Apple Patent | Method And Appartus For Data Capture And Evaluation Of Ambient Data

Patent: Method And Appartus For Data Capture And Evaluation Of Ambient Data

Publication Number: 20200081524

Publication Date: 20200312

Applicants: Apple

Abstract

The invention relates to an apparatus and a method for capturing data from the ambient data of an environment (12) of a user by means of a scene image recording device (16) and for the evaluation of the acquired ambient data by means of an evaluation device (22). Here a spatial and/or temporal selection (36a, 36b, 36c) is made which concerns an acquisition of the ambient data by means of the scene image recording device (16) and/or a transmission of the ambient data from the scene image recording device (16) to the evaluation device (22) and/or an evaluation of the ambient data by the evaluation device (22). Furthermore, the selection (36a, 36b, 36c) is made as a function of at least one captured, temporally variable first parameter (30, 32) in order to make this selection on the basis of its relevance and thus to make possible a reduction of the data volume by reduction measures restricted to less relevant data.

DESCRIPTION

[0001] The invention is based on a method for the data capture of ambient data from an environment of a user by means of a scene image recording device and for the evaluation of the acquired ambient data by means of an evaluation device. Furthermore, the invention is based on a corresponding apparatus with a scene image recording device for the data capture of ambient data from an environment of a user and with an evaluation device for evaluating the acquired ambient data.

[0002] Numerous possible applications are known from the prior art, in which a data capture of ambient data, for example, in the form of images, by means of a scene camera, as well as their evaluation plays a major role. For example, augmented reality systems, such as augmented reality glasses, can have a scene camera or ambience camera arranged at the front which takes pictures of the user’s environment. Furthermore, computer-generated objects can be superimposed on the user-perceived reality by means of such glasses, which in particular can be related to objects of the real environment. For example, additional information about objects in the environment can be superimposed by the glasses. To make this possible, the images captured by the scene camera are evaluated and searched for existing or specific objects. If such objects are found in the image recordings, the corresponding information can be overlaid by the glasses.

[0003] In combination with mobile eye-tracking systems, there are numerous other possible applications. If, for example, such glasses also incorporate an eye tracker, the eye-tracking data can be matched with the recorded scene-image data to determine, for example, where in his environment the user is currently looking, in particular, at which object in his environment. In addition, registration processes can also still be used here which make it possible to map an image recording onto a reference image which was recorded, for example, from a different perspective. Such registration processes can be used to accumulate gaze direction data over time as well as covering multiple users in a simple manner by transferring said data to a common reference image. With registration processes, particular objects, significant areas, or significant points in the reference image can be, for example, defined, such as patterns, edges, or points of intersection of edges, which during evaluation of the scene image are searched for in this and identified. By means of the correspondence between the points or areas identified in the scene image and those in the reference image, a transformation can be derived that maps the scene image onto the reference image. This transformation can, for example, be used in the same way in order to map onto the reference image the user’s point of regard in the scene image.

[0004] In all these methods, in which scene images or scene videos are recorded and evaluated, there is the problem that very large amounts of data may be incurred. Both the data transmission and its evaluation is thus time-consuming and requires a lot of computing capacity. As a result, real-time systems can only be realized to a limited extent or not at all, at least if a high quality is desired or required either with regard to the image recordings themselves or also in the evaluation, for example, in order to make high reliability possible in the recognition of objects in images. Furthermore, possibilities for data reduction are also known, such as compression methods. However, these again have the great disadvantage that such data reduction also leads to information being lost which, in particular, can be very relevant and thus in turn lower to an extreme extent the quality of such a system or apparatus.

[0005] It is therefore an object of the present invention to provide a method and an apparatus for the data capture and evaluation of ambient data, which will enable a reduction in data quantities while at the same time minimizing the loss of relevant data.

[0006] This object is achieved by a method and an apparatus having the features of the independent claims. Advantageous embodiments of the invention may be found in the dependent claims. The inventive method for the data capture of ambient data from the environment of a user by means of a scene image recording device, such as, for example, a scene camera, and for evaluation of the recorded ambient data by means of an evaluation device is characterized in that a spatial and/or temporal selection is made, which concerns an acquisition of the ambient data by means of the scene image recording device and/or a transmission of the ambient data from the scene image recording device to the evaluation device and/or an evaluation of the ambient data by the evaluation device. This selection is made as a function of at least one acquired and temporally variable first parameter, and is in particular controlled or even regulated.

[0007] By making a selection, it is advantageously possible to categorize data, for example, in terms of their relevance, specified by the first acquired parameter. Several temporal and/or spatial selections can also be made here, for example, a first selection for ambient data of the highest relevance, a second selection for ambient data of middling relevance, a third selection for ambient data of low relevance, and so on. Various reduction measures can then be advantageously limited to non-relevant or less relevant data, so that overall the amount of data can be reduced without having to forego relevant information.

[0008] In addition, such a selection can advantageously be made along the entire data path from acquisition to evaluation, so that numerous possibilities for data reduction are provided. For example, the selection concerning the acquisition of the ambient data may specify which of the image data captured by the scene image recording device are read out from an image sensor of the scene image recording device and which not, or even how often and at what rate. The selection relating to the transmission of the ambient data may, for example, specify which of the acquired data are transmitted, which not, or also in which quality, for example, compressed or uncompressed. A selection related to the evaluation may, for example, determine which of the data will be evaluated, which not, or which first. A temporal selection may specify, for example, when data are collected, read out, transmitted, or evaluated, when not, or at what rate. In this way numerous options are overall provided on the one hand for making a selection and on the other for possible ways of dealing with the selected and the unselected data, thereby providing numerous optimization possibilities with regard to bandwidth efficiency and data relevance. The invention thus makes possible a significant reduction in the base bandwidth, whereby the gain in bandwidth can again be translated into faster frame rates, faster processing, shorter latencies, lower energy or processing power requirements, simpler interfaces and less expensive components.

[0009] Here the scene image recording device can generally take the form of one or more cameras, for example, a classic 2D sensor, a so-called event-based sensor and/or even a 3D camera (for example, TOF, depth map, stereo camera, and so on).

[0010] In an advantageous embodiment of the invention, the ambient data selected according to the selection are treated in a first predefinable manner, in particular captured with the scene image recording device and/or read out from the scene image recording device and/or transmitted to the evaluation device and/or evaluated by this, and the ambient data not selected according to the selection either not treated or treated in at least a second predefinable way which differs from the first, in particular, captured again and/or read and/or transmitted and/or evaluated. As a result, advantageously different reduction measures can be applied to the selected as well as to the non-selected ambient data. For example, the selected ambient data can be acquired, transmitted and evaluated at maximum quality, while the non-selected ambient data can, for example, not be utilized at all, whereby the total amount of data can be reduced in a particularly effective way, or at least captured at lower quality, transmitted or evaluated with lower priority, which advantageously still allows use of this data while simultaneously carrying out data reduction.

[0011] In a particularly advantageous embodiment of the invention, the ambient data not selected according to the selection are reduced, in particular while the ambient data selected according to the selection are not reduced. In the case of a spatial selection, a reduction of this kind may, for example, be achieved by non-selected image areas not being recorded in the first place, transmitted to the evaluation device or evaluated by it, or by non-selected image areas being compressed, structurally reduced, for example, in their color depth, or similar. In the event of multiple cameras of the scene image recording device being available, a spatial selection can, for example, be also made by selecting one of these cameras for data capture of the ambient data. Alternatively, only selected information levels are recorded by individual or multiple cameras, read out, transferred or processed. This may concern, for example, reduction to edges, depth information, gray-scale values, certain frequency ranges. In this case, a spatial selection, in particular irrespective of whether it relates to acquisition, transmission or evaluation, has the consequence that an image reduced in terms of its data volume as compared with the originally recorded image is provided for the purpose of evaluation. In the case of a temporal selection, the reduction can be achieved, for example, by the image data either not being captured in the first place or being recorded with a low frame rate, transmitted and/or evaluated. As a result of all these measures, it is advantageously possible to reduce the total amount of data, whereby the fact that this reduction is preferably restricted to the non-selected ambient data means that the loss of information with respect to relevant data can be kept as low as possible.

[0012] Here it is particularly advantageous if, for the purpose of reduction, in particular of structural reduction, of the ambient data not selected according to the selection, the ambient data not selected according to the selection are compressed. Alternatively or additionally, the ambient data not selected according to the selection may be filtered out. Data compression can be achieved, for example, by binning or color compression, while filtering can, for example, use color filters, for example, to reduce color depth. In this way, the quantities of non-selected data can be reduced without losing the data completely. In the event that the selected data do not turn out to contain the desired information, it is still always possible to have recourse to the unselected data. Alternatively or additionally, as a way of reducing the non-selected data, provision can also be made for reducing a rate concerning the acquisition and/or transmission and/or evaluation of the ambient data not selected according to the selection in comparison with a rate concerning the acquisition and/or transmission and/or evaluation of the ambient data selected according to the selection. As long as, for example, no relevant information is expected in the image recordings, which can, for example, can be specified by the at least one parameter, depending on which this selection is made, the image capture rate can, for example, be kept low and thus also the amount of data to be acquired, transferred and ultimately evaluated–effectively in sleep mode for data capture. Similarly the transmission rate or the evaluation rate can also be reduced. Alternatively or additionally, provision can also be made, particularly in evaluation, for a lower time priority to be given to ambient data not selected according to the selection in comparison with ambient data selected according to the selection. This, for example, makes it possible for a relevant image area to be evaluated and analyzed first of all, and only when the information sought is not found, such as, for example, objects to be detected, can the non-selected data also then be evaluated. This variant saves enormous amounts of time in the evaluation since analysis can begin in image areas for which there is a high probability that the information or objects sought are to be found there. In the case of a 3D scene camera as a scene image recording device, it is also conceivable that only a part of the so-called depth map is used as a reduction measure. This part can in turn be determined or selected as a function of an eye parameter of the user as well as of one or more image characteristics, such as, for example, on basis of the determined gaze direction and its point of intersection with an object, or on the basis of vergency, the accommodation state of the eye, and so on.

[0013] The way in which the ambient data not selected according to the selection are reduced, for example, which of the above-mentioned reduction measures is applied, can either be set in advance or be determined by one or more other parameters. For example, this can be done as a function of the image characteristic. In the course of a preliminary analysis of a recorded image it can, for example, be determined whether around the area of the point of regard there are any objects at all or more objects in the image. If user is looking at, for example, a specific point on a white wall, it can be determined on the basis of the preliminary analysis that the ambient data not selected according to the selection should not, for example, be further treated at all rather than only being transferred or evaluated in compressed form. The type of reduction of ambient data can also be implemented as a function of prespecified requirements with regard to a maximum amount of data or the data rate during transmission and/or evaluation so that an appropriate compression type is selected which meets these requirements. The type of reduction can also be selected in dependence on an application or in general on the purpose of data analysis and processing. Should, for example, color information play a subordinate role here and it is good contrast or high resolution which is important instead, color filters, for example, can be selected as a reduction measure instead of using compression measures which reduce resolution as a whole. The selection parameters described for the reduction measures are here advantageous in particular when applied to structural and/or spatial reduction measures.

[0014] According to a further advantageous embodiment of the invention, the ambient data selected according to the selection can be enriched by additional data from a data source other than the scene image recording device. Enrichments of this kind may, for example, be pre-defined highlighting, in particular, relating to color or even in relation to contrast, an annotation by the user, for example, by speech input, an annotation of biometric or other user data, and/or an annotation or combination of performance data relating to an action, task or application just carried out or executed by the user.

[0015] In the evaluation of the selected ambient data these can advantageously be evaluated with further additional data and, for example, evaluated with regard to precisely that additional information. The data source can, for example, be a further capture device, for example, a voice capture device, a gesture capture device, a pulse monitor, an EEG, or even a memory device in which additional data are stored.

[0016] To now be able to suitably specify, for example, which image areas of a captured image contain relevant information or when image recordings do so, several advantageous possibilities also come into consideration for the at least one acquired and temporally variable first parameter, and will be explained in more detail below.

[0017] It is especially advantageous when the at least one first acquired parameter represents an eye parameter of at least one eye of the user. An eye tracker, for example, can be used for acquiring the eye parameter. In particular, the eye parameter can here represent a gaze direction and/or a point of regard, in particular an eye movement and/or a visual tracking movement and/or a time sequence of the point of regard, and/or eye opening state, and/or represent an item of information about a distance of the point of regard from the user, such as, for example, a convergence angle of the user’s two eyes. The invention is here based on the recognition that in particular objects or significant points, such as corners or edges of objects, attract the attention of the eye, in particular in contrast to, for example, color-homogeneous and non-structured surfaces. For example, if the user looks around in his environment, his eyes look at salient points and areas, such as corners, edges, or objects in general, entirely automatically. This can be exploited in a particularly advantageous manner for object detection in the user’s environment since it can be assumed that relevant objects, points or regions are located with very high probability at the point in the environment at which the user is currently looking. The acquired gaze direction can, for example, be compared with the ambient data captured by the scene image recording device in order to determine the point of regard of the user in the corresponding scene image.

[0018] Next an area around this registered point of regard in the scene image can, for example, be spatially selected and accordingly treated as a relevant image area by the first predefinable method, while image areas outside this area can be classified as irrelevant or less relevant, and thus cannot be treated for reduction of the volume of data, or if so then by the second predefinable method. Even information about a distance of the point of regard from the user, such as, for example, from an angle of convergence of the two eyes or an accommodation state of the at least one eye, can advantageously be used for making a spatial selection, in particular also for making a three-dimensional data selection. For example, in the case of a 3D scene camera as scene image recording device, only a part of the so-called depth map can be used, in particular even only for the selected three-dimensional area.

[0019] The point of regard of the user in the scene image, in particular as a 2D point of regard or even a 3D point of regard, can here in particular be determined by the scene image recording device and the eye tracker working in synchrony to determine the gaze direction and/or the point of regard. For example, the eye tracker can at the same time make an image recording of one eye of the user and from this determine the gaze direction in which the scene image recording device is making a corresponding image recording of the environment in order to collect ambient data. Here the scene image recording device is also preferably so arranged and/or designed that the visual field of the scene image recording device mostly coincides with the visual field of the user or at least with a possible visual field of the user, and in particular encompasses it completely. It is here particularly advantageous when, for example, the scene image recording device here represents part of a head-mounted device which in addition also includes the eye tracker. When the user makes a head movement, the scene image recording device advantageously also moves along with it, thereby keeping said scene image recording device advantageously directed at all times in the direction of the user’s field of view. But developments would also be conceivable in which the scene image recording device were arranged, not mounted on the user’s head or any other part of the user’s body, but rather fixed in one location, for example. The scene image recording device may comprise, for example, one or more cameras, which then preferably cover the largest possible solid angle range of a space.

[0020] In this case, for example, a position of the user or of his head in relation to the scene camera coordinate system could also be determined by the scene image recording device and the correspondingly determined gaze direction could also be converted into the scene camera system. This is particularly advantageous when the scene camera system is installed at a distance from the user and the bandwidth for data transmission is low or unreliable. In such cases, the invention allows the use of the available bandwidth for the information essential to the user.

[0021] The gaze direction corresponding to an image recording of the scene image recording device at a specific time does not necessarily have to be determined on the basis of image data picked up by the user’s eye at the same time. The gaze direction and/or the resulting point of regard can, for example, also be predicted on the basis of one or more previously recorded image recordings of the eye, for example, by means of a Kalman filter or other methods. The idea behind prediction of the point of regard here consists of being able to subdivide eye movements or gaze movements into saccades and fixations, in particular, moving and non-moving fixations. A saccade represents the changeover between two fixations. During such a saccade, the eye does not receive information, but only does so during a fixation. Furthermore, such a saccade follows a ballistic eye movement, so that, for example, by detecting initial values of such a saccade, such as initial velocity, initial acceleration and its direction, it is possible to determine the time point and location of the end point of such a saccade, which ultimately terminates in a fixation, and thus permits predictions. Such predictions of the point of regard can advantageously also be used in the present case in order, for example, to predict the gaze direction or the point of regard for a time at which the scene image recording device then records a corresponding ambient image. In particular, in the case of saccades, the end point can be determined and used for the next temporal and/or spatial selection at the end of the saccade. Latencies can thus advantageously also be shortened. A spatial selection can then be made by shifting the area to be selected or a second spatial area to the point of regard or to the possible points of regard in a prediction window.

[0022] However not only the point of regard or the gaze direction can be advantageously used to select relevant and non-relevant data, but also, for example, a viewing pattern or eye movements or temporal point of regard sequences or characteristic eye movement sequences, such as the saccades and fixations just described. Such viewing patterns can preferably be used here particularly advantageously for a temporal selection since, as described, a user does not take in any ambient data during a saccade. Thus, even the points of regard in an image captured by the scene image recording device during a saccade are less suitable for providing information about the presence of relevant objects, points or areas in the image. On the other hand, however, points of regard in the scene image which are to be assigned to a fixation are very suitable for supplying an indication of the presence of relevant objects, points or areas. Since these two states can be distinguished on the basis of the characteristic eye movements for saccade and fixation, and thus can be acquired, these states are particularly well suitable for making a temporal selection with regard to relevant ambient data. For example, it may be provided that image recordings are not made of the environment unless a fixation has been detected. The image recording rate can also be reduced during a non-moving fixation as compared with a moving fixation, for example, can even be restricted to one or a few fixations during a captured or predicted fixation phase since during a non-moving fixation even the point of regard of the user does not change with regard to his environment. Point of regard sequence movements can also be recognized and thus a moving object be classed as being of particular importance. Conclusions can also be drawn from pupil reactions regarding the importance of certain image contents and thus support a selection for recording, transmission, or analysis. These selection and reduction measures described for image recordings can in the same way also be applied additionally or alternatively for reading the image data, transmitting and evaluating the data.

[0023] The same also applies to the eye opening state which is also particularly suitable for making a temporal selection for capturing, reading, transmitting and/or evaluating the ambient data. Since the eye, for example, during blinking, cannot supply any information about relevant image areas, it can be provided that only when the eye is open will image recordings be made of the environment by the scene image recording device or only then will these data be transmitted or evaluated, while when a blink is detected, image recordings, whose transmission or evaluation can be dispensed with, or carried out at a lower time rate, or transmitted in compressed form, or reduced by other reduction measures will be so treated.

[0024] An acquired eye parameter will thus supply a great deal of advantageous information about where and when relevant information is present in the surroundings of a user or in the corresponding images recorded by the scene image recording device. This advantageously makes it possible to make and even to control the temporal and/or spatial selection in such a way that on the one hand data volumes can be particularly effectively reduced and on the other hand the loss of relevant data is cut to a minimum.

[0025] Alternatively or additionally, it can also be arranged that the at least one acquired parameter represents a single image characteristic of an image recorded by the scene image recording device during the capture of ambient data and/or a change in the image characteristic in relation to at least one previously recorded image. It is, for example, especially advantageous here to use as image characteristic the image content of the captured image or the change in the image content of the captured image with reference to a previously recorded image as the at least one first parameter, since, if the image content has changed not at all or only slightly in comparison with a previously captured image, it will be possible, for example, to access previously determined results without having to evaluate the newly captured image. For example, it can also be arranged that as long as the image content has not changed significantly, images will be acquired, transmitted or evaluated at a lower rate or frequency, which in turn makes for enormous savings in data. This image content comparison can be performed, for example, in the course of a pre-processing action, in particular before the image data are transmitted to the evaluation device and evaluated by the same. In contrast to a detailed image analysis, such an image content comparison can be carried out in a way which takes considerably less time and is less computationally intensive. Such an image content comparison can relate to the entire captured scene image or just to a portion of it, as again, for example, to a previous spatially selected area around the determined point of regard of the user. On the basis of a result of such comparison it can then be decided, for example, whether the image data recorded will even be transmitted to the evaluation device or evaluated by it. Other advantageous image characteristics which can be used as the at least one first parameter include, for example, spatial frequencies in the recorded scene, a contrast or contrast curves, the presence of objects, areas, or significant points in the image, a number of objects, areas or points present in the image, or even the arrangement of objects, areas, points, structures and so on present in the image. Such image parameters or image characteristics can be used advantageously in particular to make or control a spatial selection, which will be explained in more detail later.

[0026] In a further advantageous embodiment of the invention, the at least one first parameter represents a user input or a detected user characteristic or even any other external event from other signal sources or input modalities. Such parameters may alternatively or additionally also be used to trigger, for example, the recording, the transmission or the transfer and/or analysis or evaluation of individual images or image sequences, and in particular also to control or regulate them. To capture user input, conventional control elements such as buttons, mouse, and so forth may be used, as also gesture detection or the like. This allows the user to actively signal, for example, when interesting or relevant objects fall within his field of view or when he is looking at them. User characteristics may be captured, for example, by detecting the movements of a user, his gestures, by EEG signals or the like. Such characteristics can also give information, whether or not interesting objects are or are not to be found right then in the field of view of the user. It is particularly advantageous here to provide such parameters for a temporal selection of relevant data.

[0027] In a further advantageous embodiment of the invention, the spatial selection determines which area of the surroundings is captured by the scene image recording device as the ambient data, in particular in the first predefinable way and/or read out from the scene image recording device and/or transmitted to the evaluation device and/or evaluated by the evaluation device. Along the entire data path from acquisition to evaluation, it is thus advantageously possible to select data on a spatial basis, thereby characterizing the relevant data.

[0028] According to a further advantageous embodiment of the invention, the spatial selection is made in such a way–as a function of a captured point of regard of the user–that the area encompasses the point of regard. As has already been described, the point of regard is particularly suitable for the ability to select between relevant and non-relevant or less relevant data. The point of regard is thus particularly well suited as the acquired parameter, as a function of which the spatial selection is made and possibly also time-controlled.

[0029] In this context, it may further be provided that the size of the area is fixed in advance, in other words, is not variable but is constant. For example, the point of regard of the user can be determined in a corresponding image of the scene image recording device and then an area defined in terms of its size may be selected around this point of regard as the relevant data. This area can, for example, be specified in advance by a predefined fixed radius around the point of regard or as a fixed image portion with respect to the entire recorded scene image. This represents a particularly simple, less computationally intensive and above all time-saving way to select and define the area with the relevant image data.

[0030] Alternatively it can also be arranged that the size of the area is defined or controlled as a function of at least one second parameter. This provides particularly flexible options for selecting the relevant image data. This allows, for example, an adaptive adjustment in order to distinguish even better between relevant and non-relevant data around the point of regard. Suitable as this second parameter is once again and above all an image characteristic of an image acquired during the acquisition of the ambient data by means of the scene image recording device and/or a measure for the accuracy and/or dynamics of the detected point of regard of the user and/or at least one device parameter, such as transmission quality, latencies or performance of the processing device of an apparatus comprising the scene image recording device and/or the evaluation device and/or a size of an object located with at least a partial overlap of the point of regard of the user in an image captured during recording of the ambient data by the scene image recording device. If, for example, the second parameter represents the image characteristic, then on the basis of, for example, the characteristic of the image content, such as, for example, spatial frequency around the point of regard, the number or unambiguousness of the objects or relevant points, object clusters, feature clusters, contrast intensity around the point of regard or objects detected behind, in front of or around the point of regard can be used in order to determine or control the size and also the borders of the area to be defined. This makes it possible, for example, to define the area in such a way that an entire object at which the user is currently looking can always be covered or, for example, a contiguous area, or everything from the point of regard as far as the next edge (start burst), and so on. It is therefore particularly advantageous to define or control the size of the area depending on a size of an object on which the point of regard is resting or which is at least in a predetermined proximity to the point of regard of the user so that in particular the entire object or even an object group is always co-selected. This advantageously increases the probability that the relevant information to be acquired is also completely covered by the selected area. It is also particularly advantageous to provide the measure for the accuracy of the determined point of regard of the user to be the second parameter. For example, if the eye-tracking quality is poor, it may be that the determined point of regard deviates greatly from the actual point of regard of the user. However, so that as far as possible all relevant data are detected or selected, it is particularly advantageous, in the case of less accurate determination of the point of regard, to increase the area to be selected around the point of regard in contrast to the case of higher accuracy in determination of the point of regard. The accuracy of the determined point of regard can be calculated or estimated by known methods, such as from the viewing quality of the image taken by the eye tracker, the temporal scatter of point of regard values, and so on.

[0031] Even the dynamics of the point of regard can also be taken into account advantageously in the control of the size of the area. If the point of regard has a high dynamic over time, in other words, it moves or jumps swiftly within a large surrounding area, the size of the area to be selected can then be selected correspondingly larger. Various other device parameters can also be considered in defining the size of the area. For example, given an overall low performance of the apparatus or low computational power, low transmission bandwidth and so on, the area to be selected may be selected correspondingly smaller in size in order to reduce the amount of data to be transmitted or evaluated, thereby shortening latencies or keeping them within predefinable limits. In contrast to this, with higher performance of the apparatus a correspondingly larger area may also be selected. Such performance parameters can, for example, affect both transmission and evaluation as well as various other components of the apparatus. It can also, for example, be provided that the user himself or another person can specify to the system this second parameter for defining the size of the area. The user himself can thus set his own priorities regarding time efficiency or data reduction, and the quality of the result. The larger the area selected, the more likely it is that all of the relevant information from this area will be covered, while the smaller this area is selected, the fewer data must be read out, transmitted and/or evaluated.

[0032] In a further advantageous embodiment of the invention, the temporal selection determines when, in particular in the first predefinable way, an area of the environment is captured by the scene image recording device as ambient data and/or read out from the scene image recording device and/or transmitted to the evaluation device and/or evaluated by the evaluation device. In the same way as with spatial selection, a selection of data can advantageously be made with regard to the entire data path. Numerous possibilities for data reduction are thus provided, for example, already during data acquisition, or not until transmission or ultimately, not until evaluation of the acquired data.

[0033] In this case, it is furthermore particularly advantageous if the temporal selection is made as a function of the at least one first parameter such that only then or in the first predefinable way images are captured, for example, at a higher temporal rate, uncompressed, unfiltered, and so on and/or recorded image data are read out and/or evaluated by the evaluation device when the at least one first parameter fulfills a prespecified criterion. In data selection there is therefore the possibility of either not further treating non-selected data at all, in particular of not even capturing said data in the first place, or reading, transmitting and/or processing these data in a reduced manner, such as by compression or filtering or less frequently. Via the at least one acquired first parameter, time-based control of the selection can thus advantageously be made so that data volumes can again be reduced by data classed as less relevant not even being processed or at least being processed at lower quality on account of reduction without thereby affecting the quality of relevant data.

[0034] One particularly advantageous embodiment of the invention further envisages that the prespecified criterion that what is captured is that a viewing pattern and/or an eye movement and/or a visual tracking movement and/or a fixation of the eye which is captured and/or predicted as the at least one first parameter, has a prespecified characteristic. As described earlier, the presence of a fixation of the eye makes it possible to deduce the existence of relevant ambient data; even slow and continuous eye movements can, for example, suggest that the eye is tracking a moving object, which can be categorized as relevant ambient data, and so on. Alternatively or additionally, the prespecified criterion can be that on the basis of the opening state of the eye as the at least one first parameter it is understood and/or predicted that at least one eye of the user is open. It is precisely with methods in which the point of regard of the user in relation to objects in his environment is to be analyzed and evaluated that a particularly advantageous way is found of reducing data volumes by not even considering image recordings during which the user closed his eyes, for example, by blinking, since they contain no relevant information. Alternatively or additionally, the prespecified criterion can also be that on the basis of the image characteristic as the at least one first parameter it is recognized that there has been a change in at least one part of the image content in comparison with at least one part of the image content of a previously recorded image which exceeds a prespecifiable level. If the image content does not change or not significantly, the newly captured image will not contain additional, new or relevant information either, which means that the amount of data can also be advantageously reduced. Alternatively or additionally, the prespecified criterion can also be that a user input is detected as the at least one first parameter. This allows the user to inform the system himself when particularly relevant information lies within his field of view. Alternatively or additionally, it can be also arranged that the user passively provides information about the existence of relevant information in his field of vision, for example, by a prespecified user state being detected or predicted as the at least one first parameter on the basis of a user characteristic, such as, for example, EEG signals. Even user behavior, such as, for example, gestures or the like, can be analyzed to provide information about the existence of relevant information in the user’s environment. The criteria mentioned can be provided either singly or even in combination, thus affording numerous possibilities of being able to characterize situations in which relevant information, in particular relevant ambient data are available, and situations in which such relevant ambient data are not available.