空 挡 广 告 位 | 空 挡 广 告 位

Microsoft Patent | Detection of pen location relative to an electronic device

小编 映维   |   分类：Microsoft   |  

Patent: Detection of pen location relative to an electronic device

Drawings: Click to check drawins

Publication Number: 20210018990

Publication Date: 20210121

Applicant: Microsoft

Abstract

Signals are transmitted from a plurality of regions of an electronic device. Location information is then received from an electronic pen in proximity to the electronic device. The location information is determined by the electronic pen and corresponds to a location of the electronic pen relative to the screen of the electronic device. The location information is determined using one or more of the transmitted signals received by the electronic pen. Aa display on the screen of the electronic device is controlled based at least in part on the received location information. A more efficient pen operating state is thereby provided that improves the user experience by more accurately and reliably determining the pen location in space.

Claims

  1. A display arrangement comprising: a first electronic device comprising a screen; a second electronic device comprising a screen, wherein the first and second electronic devices are positioned side by side; the first and second electronic devices each further comprising at least one processor and at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the first and second electronic devices to at least: transmit signals from a plurality of regions of the first electronic device; transmit signals from a plurality of regions of the second electronic device, wherein signals transmitted from the first electronic device have at least one of a different frequency or amplitude than signals transmitted from the second electronic device; receive location information from an electronic pen in proximity to the first and second electronic devices, the location information determined by the electronic pen using one or more of the transmitted signals received by the electronic pen, the electronic pen configured to recognize different signals transmitted from the first and second electronic devices and decode the different signals to determine the location information indicating that the electronic device is at an end of a screen of the first electronic device or at an end of a screen of the second electronic device; and control a display on the screen of the first electronic device or a display on the screen of the second electronic device based at least in part on the received location information.

  2. The display arrangement of claim 1, wherein the location information causes the first electronic device or the second electronic device to be an active device based on the determination that the electronic pen is at the end of the screen of the first electronic device or at the end of the screen of the second electronic device, and the other one of the first and second electronic device being an inactive device, and wherein a portion of the screen of the active device is caused to display information and the inactive device displays no information.

  3. The display arrangement of claim 1, wherein the location information comprises a three-dimensional (3D) location of the electronic pen relative to the first and second electronic devices to identity a direction to which the electronic pen is pointing, the direction being one of towards the first electronic device or the second electronic device.

  4. The display arrangement of claim 1, wherein the electronic pen is positioned between the first and second electronic devices and extending between the ends of the first and second electronic devices, and further comprising performing triangulation at the electronic pen based on the received signals to determine the location of the electronic pen relative to the screen of the first electronic device or the screen of the second electronic device.

  5. The display arrangement of claim 1, wherein transmitting signals comprises a time division transmission wherein signals are transmitted from each region of the plurality of regions of the first and second electronic devices at different times.

  6. The display arrangement of claim 1, wherein transmitting signals from the first and second electronic devices comprises transmitting the signals from the plurality of regions of the first and second electronic devices simultaneously.

  7. The display arrangement of claim 1, wherein the screens of the first and second electronic devices each have a plurality of electrodes and further comprising driving subsets of electrodes of the plurality of electrodes corresponding to each region of the plurality of regions only at the ends of the screens of the first and second electronic devices.

  8. The display arrangement of claim 1, further comprising using a lookup table stored at the electronic pen to determine the location of the electronic pen relative to the screen of the first electronic device or the screen of the second electronic device based at least on one or more of the transmitted signals.

  9. A method to determine a location of an electronic pen, the method comprising: transmitting signals from a plurality of regions of a first electronic device; transmitting signals from a plurality of regions of a second electronic device, wherein the first and second electronic device are positioned side by side and wherein signals transmitted from the first electronic device have at least one of a different frequency or amplitude than signals transmitted from the second electronic device; receiving location information from an electronic pen in proximity to the first and second electronic devices, the location information determined by the electronic pen using one or more of the transmitted signals received by the electronic pen, the electronic pen configured to recognize different signals transmitted from the first and second electronic devices and decode the different signals to determine the location information indicating that the electronic device is at an end of a screen of the first electronic device or at an end of a screen of the second electronic device; and controlling a display on a screen of the first electronic device or a display on a screen of the second electronic device based at least in part on the received location information.

  10. The method of claim 9, wherein the location information causes the first electronic device or the second electronic device to be an active device based on the determination that the electronic pen is at the end of the screen of the first electronic device or at the end of the screen of the second electronic device, and the other one of the first and second electronic device being an inactive device, and wherein a portion of the screen of the active device is caused to display information and the inactive device displays no information.

  11. The method of claim 9, wherein the location information comprises a three-dimensional (3D) location of the electronic pen relative to the first and second electronic devices to identity a direction to which the electronic pen is pointing, the direction being one of towards the first electronic device or the second electronic device.

  12. The method of claim 9, wherein the electronic pen is positioned between the first and second electronic devices and extending between the ends of the first and second electronic devices, and further comprising performing triangulation at the electronic pen based on the received signals to determine the location of the electronic pen relative to the screen of the first electronic device or the screen of the second electronic device.

  13. The method of claim 9, where the first and second electronic devices are whiteboards positioned with the end of the screen of each abutting one another.

  14. The method of claim 9, wherein transmitting signals comprises a time division transmission wherein signals are transmitted from each region of the plurality of regions of the first and second electronic devices at different times.

  15. The method of claim 9, wherein transmitting signals from the first and second electronic devices comprises transmitting the signals from the plurality of regions of the first and second electronic devices simultaneously.

  16. The method of claim 9, wherein the screens of the first and second electronic devices each have a plurality of electrodes and further comprising driving subsets of electrodes of the plurality of electrodes corresponding to each region of the plurality of regions only at the ends of the screens of the first and second electronic devices.

  17. The method of claim 9, wherein transmitting signals comprises transmitting uplink signals in response to detection of the electronic pen within a proximity threshold of the screen of the first electronic device or the screen of the second electronic device.

  18. The method of claim 9, further comprising using a lookup table stored at the electronic pen to determine the location of the electronic pen relative to the screen of the first electronic device or the screen of the second electronic device based at least on one or more of the transmitted signals.

  19. One or more computer storage media having computer-executable instructions that, upon execution by a processor of a first electronic device and a processor of a second electronic, cause the processors to at least: transmit signals from a plurality of regions of the first electronic device; transmit signals from a plurality of regions of the second electronic device, wherein the first and second electronic device are positioned side by side and wherein signals transmitted from the first electronic device have at least one of a different frequency or amplitude than signals transmitted from the second electronic device; receive location information from an electronic pen in proximity to the first and second electronic devices, the location information determined by the electronic pen using one or more of the transmitted signals received by the electronic pen, the electronic pen configured to recognize different signals transmitted from the first and second electronic devices and decode the different signals to determine the location information indicating that the electronic device is at an end of a screen of the first electronic device or at an end of a screen of the second electronic device; and controlling a display on a screen of the first electronic device or a display on a screen of the second electronic device based at least in part on the received location information.

  20. The one or more computer storage media of claim 19, wherein the location information causes the first electronic device or the second electronic device to be an active device based on the determination that the electronic pen is at the end of the screen of the first electronic device or at the end of the screen of the second electronic device, and the other one of the first and second electronic device being an inactive device, and wherein a portion of the screen of the active device is caused to display information and the inactive device displays no information.

Description

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application is a continuation application of and claims priority to U.S. patent application Ser. No. 16/125,649, entitled “DETECTION OF PEN LOCATION RELATIVE TO AN ELECTRONIC DEVICE,” filed on Sep. 7, 2018, the disclosure of which is incorporated herein by reference in its entirety.

BACKGROUND

[0002] Electronic devices, such as smart boards, personal computers, laptops, mobile phones, and the like are increasingly equipped with screens or similar interfaces that enable a user to provide input by writing or drawing with a stylus, pen, or other pen-like device. For instance, wall-mounted or roller-stand-mounted interactive whiteboards with multi-touch and multi-pen capabilities have become increasing popular. Other electronic devices, such as tablet computers, which include touch screens as the primary user interface, have also become popular alternatives to conventional laptops with keyboards, mice, etc. The ability to draw on the screen offers flexibility regarding possible inputs.

[0003] However, while some of these devices are able to detect the proximity of an electronic pen to a screen of the devices (e.g., when the pen is in proximity, the device becomes aware of the device presence), the position of the device relative to the screen (e.g., right side or left side) if very difficult or impossible to determine. For example, some systems use a received signal magnitude to detect pen presence. But, this signal magnitude is not indicative of the pen position relative to the device. Specifically, because signal magnitude drops slowly, it is hard to determine a position of the pen, particularly if the pen is located on the side of the device. Moreover, the equi-capacitance to the device surfaces where the pen can reside around the device can lead to ambiguity with respect to the pen’s actual position, and therefore a desired pen operation.

SUMMARY

[0004] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0005] A method comprises transmitting signals from a plurality of regions of an electronic device. The method further comprises receiving location information from an electronic pen in proximity to the electronic device. The location information is determined by the electronic pen and corresponds to a location of the electronic pen relative to a screen of the electronic device. The location information is determined using one or more of the transmitted signals received by the electronic pen. The method also comprises controlling a display on the screen of the electronic device based at least in part on the received location information.

[0006] Many of the attendant features will be more readily appreciated as the same becomes better understood by reference to the following detailed description considered in connection with the accompanying drawings.

DESCRIPTION OF THE DRAWINGS

[0007] The present description will be better understood from the following detailed description read in light of the accompanying drawings, wherein:

[0008] FIG. 1 illustrates a system including an electronic device and an associated pen according to an embodiment;

[0009] FIG. 2 illustrates a transmission configuration according to an embodiment;

[0010] FIG. 3 illustrates another transmission configuration according to an embodiment;

[0011] FIG. 4 illustrates a block diagram of an electronic device according to an embodiment;

[0012] FIG. 5 illustrates a pen according to an embodiment;

[0013] FIG. 6 illustrates a diagram of interactions between a pen/device interface and a pen according to an embodiment;

[0014] FIG. 7 illustrates a flow chart of a method for determining the location of an electronic pen according to an embodiment;

[0015] FIG. 8 illustrates another flow chart of a method for determining the location of an electronic pen according to an embodiment; and

[0016] FIG. 9 illustrates a computing apparatus according to an embodiment as a functional block diagram.

[0017] Corresponding reference characters indicate corresponding parts throughout the drawings. In the figures, the systems are illustrated as schematic drawings. The drawings may not be to scale.

DETAILED DESCRIPTION

[0018] The computing devices and methods described herein transmit signals from different parts of a screen of an electronic device (e.g. electronic smartboard, such as the Microsoft Surface Hub.RTM.) to allow an electronic pen in proximity to the interface surface of the electronic device to detect or determine a position of the electronic pen relative to the electronic device. In some examples, an active pen is thereby able to know a position with respect to an inking device. As a result, an operational behavior of the pen can be change, such as changes in power modes (idle/active) and operation cycles (searching for other devices).

[0019] Touch sensors transmit beacons (uplink) signals that can be received by the pens to indicate to the pens the device presence and communication protocol. In some examples, having the beacons transmitted from different parts of the device, either at different times, or simultaneously using different signals, enables the receiving pen to determine a three-dimensional (3D) position of the pen relative to the device. Thus, by transmitting the uplink signal, for example, from only one part of the screen (and toggling over time to the other parts) enables the pen’s 3D position relative to the device to be extracted and determined by the pen (e.g., the pen decodes signals received from different parts of the electronic device to determine position). Alternatively, by transmitting different signals (e.g., Tx, Tx’, Tx”, Tx”’ having different information) from different parts of the screen simultaneously, the same location information can be determined. As a result, a more efficient pen operating state is thereby provided that improves the user experience. In this manner, when a processor is programmed to perform the operations described herein, the processor is used in an unconventional way, and allows for the more efficient user input with the device.

[0020] FIG. 1 illustrates a system 100 including an electronic device 102 and an associated pen 104 according in one example. The electronic device 102 and pen (or pen device) 104 are associated or linked such that the electronic device 102 and the pen 104 respond to each other. In one example, the pen 104 is uniquely associated with the electronic device 102 such that other pens and/or similar devices do not interact or interact differently with the electronic device 102.

[0021] The electronic device 102 includes a screen 106 defining a screen interface and is configured in one example as an electronic interactive smartboard. The screen interface receives input via touch sensor(s), pressure sensor(s), or the like. The input can be in the form of shapes, expressions, symbols, handwriting, etc. In one example, the pen 104 is used to touch, contact, depress, or otherwise interact with the screen 106 in order to provide input to the electronic device 102.

[0022] Additionally, the electronic device 102 includes one or more transmitters (not shown in FIG. 1) configured to transmit uplink signals, which in some examples are electrodes of the screen interface. In one example as illustrated in FIG. 2, the uplink signal is transmitted from only one part of the screen (and toggled over time to the other parts), which enables the 3D position of the pen 104 to be determined relative or with respect to electronic device 102 (e.g., location in space in front of the screen 106) by the pen itself. In the illustrated example, the uplink signal is transmitted from four different regions 200, each at a different time, such as sequentially from T.sub.1 then T.sub.2 then T.sub.3 then T.sub.4. It should be appreciated that additional or fewer regions 200 can be provided and the order and timing of where the uplink signal is transmitted can be varied.

[0023] In another example, as shown in FIG. 3, different uplink signals (illustrated as Tx, Tx’, Tx”, Tx”’) are transmitted from the different regions 200 of the screen 106 simultaneously. For example, each of the signals in some examples have a different frequency or amplitude. However, any types of signals having different information can be used to allow the pen 104 to determine position information relative to the electronic device 102. It should be noted that some or all of the transmission schemes illustrated in FIGS. 2 and 3 can be combined in some examples. Thus, spatial and/or temporal variations in the transmission of the signals can be implemented in various examples to allow the location of the pen 104 to be determined by the pen 104. That is, a pen 104, which is a small object with one spatial receiver, in some examples, is able to determine a location of the pen 104 relative to the electronic device 102.

[0024] The uplink signals in various examples are employed to detect the presence of the pen 104, as well as to allow the position of the pen 104 relative to the screen 106 of the electronic device 102 (e.g., in proximity to one of the regions 200) to be determined directly by the pen 104. Moreover, in some examples, one or more communication channels are established between the electronic device 102 and the pen 104, using communication techniques in the electronic pen technology, to enable uplink and downlink communication for bi-directional data exchange between the electronic device 102 and the pen 104. Thus, the uplink signal can be any type of signal transmitted from the electronic device 102 and detectable by the pen 104.

[0025] The pen 104 in one example is an active electronic pen (e.g., an active stylus) that includes electronic components that enable the pen 104 to interact with the electronic device 102, a user of the pen 104, other electronic devices, etc. For instance, in some examples, the pen 104 includes a wireless interface that enables the pen 104 to communicate wirelessly (via Wi-Fi, cellular, BLUETOOTH.RTM. short-range wireless communication protocol, other radio frequency communications, etc.) with the electronic device 102, even when the pen 104 is not in contact with the electronic device 102. Further, the pen 104 includes one or more of buttons, switches, and/or other input interfaces in some examples, which a user of the pen 104 uses to interact with the pen 104 and/or electronic components of the pen 104. Additionally, or alternatively, the pen 104 can include pressure sensors, motion sensors, accelerometers, gyroscopic sensors, or the like that enable the detection of motion, direction, angle, user input, gestures, etc. of the pen 104.

[0026] In operation, a proximity and relative position (e.g., XY position) of the pen 104 to the electronic device 102 is determined. The pen 104 is configured to receive one or more signals from the electronic device 102 to allow the pen 104 to determine a location of the pen 104 relative to the electronic device 102. In some examples, a detection of pen proximity to the electronic device 102 includes interpreting a signal strength of a wireless signal from the pen 104 as an indicator of pen proximity to thereby initiate signals being sent by the electronic device 102 to the pen 104 for use by the pen 104 to determine the location of the pen 104 (e.g., XY location of the pen 104 in front of the electronic device 102). For instance, the signal strength of the wireless signal is generally stronger when the pen 104 is close to a portion of the electronic device 102 that is transmitting the signal (e.g., uplink signal) and the signal strength of the wireless signal becomes weaker as the pen 104 moves away from the electronic device 102. Alternatively, or additionally, the pen 104 can also transmit, to the electronic device 102, information describing pen motion, pen direction, pen angle, etc. that can be used by the electronic device 102 to determine information relating to the pen 104.

[0027] In one example, when the electronic device 102 detects a proximity of the pen 104 by, at least in part, detecting a wireless signal from the pen 104, the electronic device 102 initiates a pen 104 location detection process, wherein signals are sent to the pen 104 as described herein to allow the pen 104 to determine a location thereof relative to the electronic device 102. In some examples, signal strength of the wireless signal received by the pen 104 can be used to determine pen location information.

[0028] The pen location detection process is initiated in some examples using a pen proximity threshold defined to include a signal strength threshold such that, when the signal strength threshold is crossed, the electronic device 102 determines that the pen 104 is within the pen proximity threshold and then begins transmission of the uplink signal(s) to allow the pend 104 to determine a position or location of the pen 104 relative to the screen 106 of the electronic device 102, such as where along or in front of the electronic device 102 the pen 104 is located. It should be appreciated that the initiation of the pen location determination process can be performed in many different ways, such as based on a user depression of a button on the pen 104 when in proximity to the electronic device 102, the electronic device 102 periodically transmitting signals therefrom, etc.

[0029] FIG. 4 illustrates a block diagram of the electronic device 102 in one example. The electronic device 102 includes a user interface 400, an operating system 402 that includes a pen location handler 404 and pen-compatible control(s) 406, application(s) 408, and a network interface 410. The user interface 400 further includes an input interface 412 and an output interface 414.

[0030] In one example, the user interface 400 includes a touch screen, such as a smartboard touch screen. The input interface 412 is some examples includes a layer or portion of the touch screen that detects the location of contact, depression, or the like on the touch screen. Contact on the touch screen, whether by a user’s finger, the pen 104, other types of stylus, or the like, is detected by the input interface 412 and interpreted as input to the user interface 400. The output interface 414 in some examples includes a layer or portion of the touch screen that displays, renders, or otherwise outputs information to a user of the electronic device 102. The output interface 414 can display colors, shapes, letters, or the like to communicate output information to a user of the electronic device.

[0031] Alternatively, or additionally, the input interface 412 receives input from a pen device (e.g., the pen 104) linked to the electronic device 102 as described herein. The pen 104 and electronic device 102 are in communication via the network interface 410 of the electronic device 102.

[0032] The input interface 412 can include other interfaces, such as keyboards, mice, switches, buttons, microphones, cameras, motion detection, etc. in some examples. These components of the input interface 412 further enable a user to input information into the electronic device 102. For instance, a camera associated with the input interface 412 can detect a user’s gestures and interpret the gestures as a form of input information. In another example, the camera and input interface 412 are associated with an augmented reality device and/or a virtual reality device.

[0033] In some examples, the output interface 414 further includes speakers, vibration components, projector components, etc. These components of the output interface 414 further enable the electronic device 102 to communicate output information to a user. For instance, a vibration component of the output interface 414 vibrates to provide a notification to the user of the electronic device 102.

[0034] The operating system 402 in one example is a software component configured to perform core software operations of the electronic device 102 and to provide a platform upon which other software components (e.g., application(s) 408, etc.) are executed. It should be understood that the operating system 402 functions according to typical methods as understood by a person of ordinary skill in the art of computer science, computer engineering, or the like.

[0035] The pen location handler 404 includes software that interacts with the user interface 400, including controlling signals sent to the pen device (e.g., the pen 104, etc.) for use in determining the location of the pen 104 in space relative to the electronic device 102. The pen location handler 404 initially can listen for communications from the user interface 400 associated with pen proximity data during use of the electronic device 102. Based on, for example, a received initiation signal, the pen location handler 404 triggers a pen event that is sent to and/or received by other software components (e.g., application(s) 408, etc.) of the electronic device 102 in some examples. As described in more detail herein, the pen location handler 404 controls generation of uplink signals by one or more transmitter(s) 416 to allow the pen 104 to determine the location of the pen 104 relative to the screen 106 of the electronic device 102. In some examples, transmitters 416 are electrodes in the electronic device that are selectively driven (e.g., radio-frequency (RF) driven) in defined areas corresponding to the regions 200 that allow the pen 104 to determine a relative location to the electronic device 102. Thus, in some examples, a subset of electrodes are driven that are located in each of the regions 200 to provide signals to the pen 104 for use in determining the location information.

[0036] The pen-compatible control(s) 406 are software components associated with the operating system 402 that cause the electronic device 102 to react to and/or interact with the pen 104. In one example, the pen-compatible control(s) 406 cause the user interface 400 to provide a user of the electronic device 102 with a writing region, information and/or guidance regarding context of the writing region, or the like. For instance, the pen-compatible control(s) 406 cause the user interface 400 to display a box on the screen containing a writing region and a prompt in or near the box describing the box as an address field, name field, signature field, search term field, or the like. Further the pen-compatible control(s) 406 can include check boxes and/or list items that expand to provide additional details based on the proximity of the pen or map controls that zoom in to a geographic area and provide a virtual writing area associated with the geographic area that the user can use to add notes about the geographic area. The location on the screen 106 of the displayed information changes in some examples based on the location of the pen 104 as determined by the pen 104.

[0037] The application(s) 408 are software components that can be installed on the electronic device 102. In one example, the application(s) 408 use the operating system 402 as a platform for executing instructions and/or providing functionality to a user of the electronic device 102. For instance, the application(s) 408 can be a word processing application, an email application, a web browser, a messaging application, a game, or the like. It should be understood that the application(s) 408 can be of any application type known to a person of ordinary skill in the art without departing from the scope of the description herein.

[0038] The application(s) 408 in some examples include a pen event handler component that is configured to receive, detect, and/or react to pen events that are triggered by the operating system. The pen event handler of the application(s) 408 cause the activation of pen-compatible controls or other operations to be performed.

[0039] The application(s) 408 in some examples lack application-specific controls and/or functionality to accept and process pen input, and instead access the one or more pen-compatible controls 406 from the operating system 402 for use within the application 406 when pen input is detected. In one example, a messaging application is not configured to handle pen input and, when a pen event is received by the messaging application, the messaging application accesses or requests to use a pen-compatible control provided by the operating system 402 to display a writing region and convert pen input into text which the messaging application then uses to communicate to a user’s contact in a message. The request can occur via an application programming interface (API) call to the operating system.

……
……
……

本文链接：https://patent.nweon.com/17534

您可能还喜欢...