CA2235030A1 - System for electronic newsgathering, documentary video, and photojournalism - Google Patents
System for electronic newsgathering, documentary video, and photojournalism Download PDFInfo
- Publication number
- CA2235030A1 CA2235030A1 CA002235030A CA2235030A CA2235030A1 CA 2235030 A1 CA2235030 A1 CA 2235030A1 CA 002235030 A CA002235030 A CA 002235030A CA 2235030 A CA2235030 A CA 2235030A CA 2235030 A1 CA2235030 A1 CA 2235030A1
- Authority
- CA
- Canada
- Prior art keywords
- camera
- computer system
- light
- eye
- viewfinder
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000004891 communication Methods 0.000 claims abstract description 20
- 230000006870 function Effects 0.000 abstract description 7
- 230000006978 adaptation Effects 0.000 abstract 1
- 239000000203 mixture Substances 0.000 description 12
- 230000007246 mechanism Effects 0.000 description 11
- 230000000007 visual effect Effects 0.000 description 10
- 238000000034 method Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 230000001404 mediated effect Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 206010016754 Flashback Diseases 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000013479 data entry Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000001627 detrimental effect Effects 0.000 description 2
- 230000003203 everyday effect Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000016776 visual perception Effects 0.000 description 2
- 235000005156 Brassica carinata Nutrition 0.000 description 1
- 244000257790 Brassica carinata Species 0.000 description 1
- 239000004429 Calibre Substances 0.000 description 1
- 208000001613 Gambling Diseases 0.000 description 1
- 206010047571 Visual impairment Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 229940000425 combination drug Drugs 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000010437 gem Substances 0.000 description 1
- 229910001751 gemstone Inorganic materials 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000011017 operating method Methods 0.000 description 1
- 229920000136 polysorbate Polymers 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 230000002459 sustained effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0176—Head mounted characterised by mechanical features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/163—Wearable computers, e.g. on a belt
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B2027/0178—Eyeglass type
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Optics & Photonics (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Computer Hardware Design (AREA)
- Studio Devices (AREA)
Abstract
A novel system for a new kind of electronic news gathering and videography is described. In particular, a wearable camera with viewfinder means, wearable computational means, and wireless communication is introduced. A journalist wearing the apparatus becomes, after adaptation, an entity that seeks, without conscious thought or effort, an optimal point of vantage and camera orientation. Moreover, the journalist can easily become part of a human intelligence network, and draw upon the intellectual resources and technical photographic skills of a large community.
Because of the journalist's ability to constantly see the world through the invention, which may also function as an image enhancement device, the apparatus behaves as a true extension of the journalist's mind and body, giving rise to a new genre of documentary video. In this way, it functions as a seamless communications medium that uses a reality-based user-interface.
Because of the journalist's ability to constantly see the world through the invention, which may also function as an image enhancement device, the apparatus behaves as a true extension of the journalist's mind and body, giving rise to a new genre of documentary video. In this way, it functions as a seamless communications medium that uses a reality-based user-interface.
Description
Patent Application of Steve Mann for SYSTEM FOR ELECTRONIC NEWSGATHERING, DOCUMENTARY
VIDEO, AND PHOTOJOURNALISM
of which the following is a specification:
FIELD OF THE INVENTION
The present invention pertains generally to a new photographic or video means and apparatus comprising a body-worn portable electronic camera system, wearable viewfinder means, and wireless communications means.
BACKGROUND OF THE INVENTION
In photojournalism, electronic news gathering (ENG), and in movie or video pro-duction, it is desirable to capture events in a natural manner with minimal inter-vention and disturbance. Current state-of-the-art newsgathering apparatus creates a visual disturbance to others and attracts considerable attention on account of the camera crew, sound crew, the bulky equipment, and the manner in which it is de-ployed and used. Even when a single camera operator is assigned to a newsgathering task, the simple gesture of bringing the camera up to the eye can disrupt the event being reported. Even if the size of the camera could be reduced to the point of being negligible (e.g. no bigger than the eyecup of a typical camera viewfinder, for example), the very gesture of bringing a device up to the eye is unnatural and attracts consid-erable attention, especially in establishments such as gambling casinos or department stores where photography is often prohibited, yet where newsworthy events frequently happen. Although there exist a variety of covert cameras which are frequently used in investigative journalism, such a camera concealed beneath the jewel of a necktie clip, cameras concealed in baseball caps, and cameras concealed in eyeglasses, these cameras tend to produce inferior images, not just because of the technical limitations imposed by their small size, but, more importantly because they lack a means of viewing the image, as well as a means of having others view the image. In typical newsgathering scenarios, one person operates the camera while another carries a mon-itor to observe the technical quality of the video images. Often there is a news truck equipped with instrumentation so that a technical staff can monitor the quality of the video signal, and report back to the camera crew by telephone or the like.
Because of the lack of viewfinder means, and the lack of analysis/communications means be-tween the camera operator and remote test equipment/technical staff, investigative video and photojournalism made with concealed cameras of the prior art suffers from poor composition and poor image/sound quality.
Accordingly, what is proposed is a wearable camera and viewfinder means for capturing video of exceptionally high compositional and artistic calibre, together with means of not only transmitting and recording said video at one or more re-mote locations, but also a means of computer-supported collaborative photojournal-ism between a camera operator with a possibly covert (or at least unencumbering) wearable camera/computer/personal-imaging system and a remote technical staff.
Furthermore, the proposed computer-supported collaborative photojournalism sys-tem allows the camera operator and the interviewer to become one person, in a new genre of cinematography/video characterized by first-person perspective.
Moreover, the proposed invention allows the interviewer (who is also the camera operator) to have, at his/her disposal a remote panel of experts on a given field.
In addition to the fact that covert versions of the apparatus can be used to create investigative documentary videos having very good composition, for everyday usage, the device need not necessarily be covert, and may, in fact, be a fashionable device that serves as a stylized icon of the reporter or photojournalist of the future.
BACKGROUND OF THE INVENTION WITH RESPECT TO PRIOR
VIDEO, AND PHOTOJOURNALISM
of which the following is a specification:
FIELD OF THE INVENTION
The present invention pertains generally to a new photographic or video means and apparatus comprising a body-worn portable electronic camera system, wearable viewfinder means, and wireless communications means.
BACKGROUND OF THE INVENTION
In photojournalism, electronic news gathering (ENG), and in movie or video pro-duction, it is desirable to capture events in a natural manner with minimal inter-vention and disturbance. Current state-of-the-art newsgathering apparatus creates a visual disturbance to others and attracts considerable attention on account of the camera crew, sound crew, the bulky equipment, and the manner in which it is de-ployed and used. Even when a single camera operator is assigned to a newsgathering task, the simple gesture of bringing the camera up to the eye can disrupt the event being reported. Even if the size of the camera could be reduced to the point of being negligible (e.g. no bigger than the eyecup of a typical camera viewfinder, for example), the very gesture of bringing a device up to the eye is unnatural and attracts consid-erable attention, especially in establishments such as gambling casinos or department stores where photography is often prohibited, yet where newsworthy events frequently happen. Although there exist a variety of covert cameras which are frequently used in investigative journalism, such a camera concealed beneath the jewel of a necktie clip, cameras concealed in baseball caps, and cameras concealed in eyeglasses, these cameras tend to produce inferior images, not just because of the technical limitations imposed by their small size, but, more importantly because they lack a means of viewing the image, as well as a means of having others view the image. In typical newsgathering scenarios, one person operates the camera while another carries a mon-itor to observe the technical quality of the video images. Often there is a news truck equipped with instrumentation so that a technical staff can monitor the quality of the video signal, and report back to the camera crew by telephone or the like.
Because of the lack of viewfinder means, and the lack of analysis/communications means be-tween the camera operator and remote test equipment/technical staff, investigative video and photojournalism made with concealed cameras of the prior art suffers from poor composition and poor image/sound quality.
Accordingly, what is proposed is a wearable camera and viewfinder means for capturing video of exceptionally high compositional and artistic calibre, together with means of not only transmitting and recording said video at one or more re-mote locations, but also a means of computer-supported collaborative photojournal-ism between a camera operator with a possibly covert (or at least unencumbering) wearable camera/computer/personal-imaging system and a remote technical staff.
Furthermore, the proposed computer-supported collaborative photojournalism sys-tem allows the camera operator and the interviewer to become one person, in a new genre of cinematography/video characterized by first-person perspective.
Moreover, the proposed invention allows the interviewer (who is also the camera operator) to have, at his/her disposal a remote panel of experts on a given field.
In addition to the fact that covert versions of the apparatus can be used to create investigative documentary videos having very good composition, for everyday usage, the device need not necessarily be covert, and may, in fact, be a fashionable device that serves as a stylized icon of the reporter or photojournalist of the future.
BACKGROUND OF THE INVENTION WITH RESPECT TO PRIOR
ART: Recent experiments, as conducted by and reported by Mann, in an MIT
techni-cal report Mediated Reality, medialab vismod TR260, (1994), (the report is available in http://wearcam.org/mediated-reality/index.html) studies the slight transforma-tions such as rotation by a few degrees or small image displacements that often give rise to a reversed aftereffect that is more rapidly assimilated by the wearer, and that such effects can often have a more detrimental effect on performing other tasks through the camera as well as detrimental flashbacks upon removal of the camera.
This work also looked into the effects of using wireless communications with a remote image processing system as a means of mediated reality. These findings suggest that merely mounting a conventional camera such as a small 35mm rangefinder camera or a small video camcorder to a helmet, so that one can look through the viewfinder and use it it hands-free while performing other tasks, will result in poor performance at doing those tasks while looking through the camera viewfinder, in addition to the obvious shortcoming of not having a technical staff or other remote collaborators available.
Part of the reason for poor performance associated with simply attaching a con-ventional camera to a helmet is the induced parallax and the failure to provide an orthoscopic view. Even viewfinders which correct for parallax, as described in U.S.
Pat. No.5692227 in which a rangefinder is coupled to a parallax error compensating mechanism, only correct for parallax between the viewfinder and the camera lens that is taking the picture, but do not correct for parallax between the viewfinder and the image that would be observed with the naked eye while not looking through the camera.
An object of the invention is to provide a viewfinder means that is suitable for long-term telepresence, computer supported collaborative photojournalism etc., such as when one may be typically wearing the camera for an entire day, looking through it all the while.
An important aspect of the proposed invention is the capability of the apparatus to mediate (augment, diminish, or otherwise alter) the visual perception of reality.
Traditional camera viewfinders often include the ability to overlay virtual objects, such as camera shutter speed, or the like, on top of reality, as described in U.S.
Pat. No.5664244 which describes a viewfinder with additional information display capability.
The proposed electronic news gathering system is related to the displays that are used in the field of Virtual Reality (VR) in the sense that both are wearable.
However, an important difFerence is that the proposed invention allows the wearer to continue to see the real world, while VR displays block out the ability to see the real world.
Displays for helmet mounted aircraft weapons aiming applications have been de-veloped, as described in U.S. Pat. No.3697154, U.S. Pat. No.3833300, U.S. Pat.
No.4081209, U.S. Pat. No.4220400. Such displays do not directly incorporate a cam-era. Although they could be used to display the output image from an electronic camera (hand-held or perhaps mounted to the same helmet), the above-mentioned problems will still exist. U.S. Pat. No.4806011 describes an eyeglass-based display of a clock or the like.
Open-air viewfinders are often used on extremely low cost cameras, as well as on some professional cameras for use at night when the light levels would be too low to tolerate any optical loss in the viewfinder. Examples of open-air viewfinders used on professional cameras, in addition to regular viewfinders, include those used on the GrafHex press cameras of the 1940s (which had three different kinds of viewfind-ing means), as well as those used on some twin-lens reflex cameras. While such viewfinders could be used in the context of the proposed invention, and would have the advantage of not inducing the problems such as flashback efFects described above, they also fail to provide an electronically mediated reality. It is an object of the in-vention to allow visual reality to be mediated in order to make certain that exposure is correct as well as to keep the wearer of the apparatus in the feedback loop of the photo compositional process by constantly providing the wearer with a video stream.
techni-cal report Mediated Reality, medialab vismod TR260, (1994), (the report is available in http://wearcam.org/mediated-reality/index.html) studies the slight transforma-tions such as rotation by a few degrees or small image displacements that often give rise to a reversed aftereffect that is more rapidly assimilated by the wearer, and that such effects can often have a more detrimental effect on performing other tasks through the camera as well as detrimental flashbacks upon removal of the camera.
This work also looked into the effects of using wireless communications with a remote image processing system as a means of mediated reality. These findings suggest that merely mounting a conventional camera such as a small 35mm rangefinder camera or a small video camcorder to a helmet, so that one can look through the viewfinder and use it it hands-free while performing other tasks, will result in poor performance at doing those tasks while looking through the camera viewfinder, in addition to the obvious shortcoming of not having a technical staff or other remote collaborators available.
Part of the reason for poor performance associated with simply attaching a con-ventional camera to a helmet is the induced parallax and the failure to provide an orthoscopic view. Even viewfinders which correct for parallax, as described in U.S.
Pat. No.5692227 in which a rangefinder is coupled to a parallax error compensating mechanism, only correct for parallax between the viewfinder and the camera lens that is taking the picture, but do not correct for parallax between the viewfinder and the image that would be observed with the naked eye while not looking through the camera.
An object of the invention is to provide a viewfinder means that is suitable for long-term telepresence, computer supported collaborative photojournalism etc., such as when one may be typically wearing the camera for an entire day, looking through it all the while.
An important aspect of the proposed invention is the capability of the apparatus to mediate (augment, diminish, or otherwise alter) the visual perception of reality.
Traditional camera viewfinders often include the ability to overlay virtual objects, such as camera shutter speed, or the like, on top of reality, as described in U.S.
Pat. No.5664244 which describes a viewfinder with additional information display capability.
The proposed electronic news gathering system is related to the displays that are used in the field of Virtual Reality (VR) in the sense that both are wearable.
However, an important difFerence is that the proposed invention allows the wearer to continue to see the real world, while VR displays block out the ability to see the real world.
Displays for helmet mounted aircraft weapons aiming applications have been de-veloped, as described in U.S. Pat. No.3697154, U.S. Pat. No.3833300, U.S. Pat.
No.4081209, U.S. Pat. No.4220400. Such displays do not directly incorporate a cam-era. Although they could be used to display the output image from an electronic camera (hand-held or perhaps mounted to the same helmet), the above-mentioned problems will still exist. U.S. Pat. No.4806011 describes an eyeglass-based display of a clock or the like.
Open-air viewfinders are often used on extremely low cost cameras, as well as on some professional cameras for use at night when the light levels would be too low to tolerate any optical loss in the viewfinder. Examples of open-air viewfinders used on professional cameras, in addition to regular viewfinders, include those used on the GrafHex press cameras of the 1940s (which had three different kinds of viewfind-ing means), as well as those used on some twin-lens reflex cameras. While such viewfinders could be used in the context of the proposed invention, and would have the advantage of not inducing the problems such as flashback efFects described above, they also fail to provide an electronically mediated reality. It is an object of the in-vention to allow visual reality to be mediated in order to make certain that exposure is correct as well as to keep the wearer of the apparatus in the feedback loop of the photo compositional process by constantly providing the wearer with a video stream.
Moreover, it is desired that the apparatus will allow the wearer to experience a com-putationally mediated visual reality, and for that experience to be shared through wireless communications networks so that the wearer may receive additional visual information, as well as be aware of modifications to visual reality that might arise, for example, as part of a communications process in a shared virtual environment.
For such compositional and interactional capabilities, a simple air-based viewfinder is inadequate.
Wearable display devices have been described, such as in U.S. Pat. No.5546099, U.S. Pat. No.5708449, U.S. Pat. No.5331333, U.S. Pat. No.4636866, and may be used as a starting point for implementing the proposed invention, through the addition of the appropriate light sensing apparatus and wireless communications means.
U.S.
Pat. No.5640221 also proposes an eye-tracking device which may be used in the context of the proposed invention.
DESCRIPTION OF THE INVENTION
The proposed invention facilitates a new form of visual art, in which the artist may capture, with relatively little effort, a visual experience as viewed from his or her own perspective. With some practice, it is possible to develop a very steady body posture and mode of movement that best produces video of the genre pertaining to this invention. Because the apparatus is lightweight and close to the head, there is not the protrusion associated with carrying a hand-held camera. Also because components of the proposed invention are mounted very close to the head, in a manner that balances the weight distribution as well as minimizes the moment of inertia about the rotational axis of the neck, the head can be turned quickly while wearing the apparatus.
This allows one to record the experiences of ordinary day-to-day activities from a first-person perspective, and, because of the proposed communications infrastructure, to convey these experiences to a remote entity. Moreover, because both hands are free, much better balance and posture is possible while using the apparatus. Anyone skilled in the arts of body movement control as is learned in the martial arts such as karate, as well as in dance, most notably ballet, will have little difficulty capturing exceptionally high quality video using the proposed invention.
With the prior art, the best camera operators tend to be very large people who have trained for many years in the art of smooth control of the cumbersome video or motion picture film cameras used. In addition to requiring a very large person to opti-mally operate such cameras, various stabilization devices are often used, which make the apparatus even more cumbersome. The proposed wearable camera invention may be optimally operated by people of any size. Even young children can become quite proficient in the use of the proposed invention, and could make excellent photojour-nalists, especially if given the capabilities of a network of adult experts.
A typical embodiment of the invention comprises one or two spatial light mod-ulators or other display means built into a pair of eyeglasses together with one or more light sensor arrays, a body-worn computer and image processing system, and a high-speed wireless communications system. Typically one or more CCD (charge coupled device image sensor arrays and appropriate optical elements comprise the camera portion of the invention. Typically a beamsplitter or a mirror silvered on both sides is used to combine the image of the viewfinder with the apparent position of the camera. The viewfinder is not j ust a means of determining the extent of coverage of the camera in a natural manner, but it is also a communications means, so that, for example, a remote expert may provide advice in the form of text superimposed inside the viewfinder. In this manner the viewfinder functions as a teleprompter, as well as a means for j udging photographic or video composition. Moreover, one or more remote experts may superimpose graphical elements in the viewfinder, such as a cursor or arrow that points to an object as depicted in the viewfinder.
In this way the wearer may collaborate with one or more remote experts on matters such as scene content, or, for example, specific details of a photographic composition. Fi-nally, the collaboration may take the form of computer graphics rendering inserted into the viewfinder, together with the images of real objects. This allows the user to experience a computer-mediated reality in which there is a much more rich form of collaborative potential.
SUMMARY OF THE INVENTION: OBJECTS AND ADVANTAGES
It is an object of this invention to provide a method of positioning a camera in which both hands are left free.
It is a further object of this invention to provide a means of acquiring a picture electronically where the spatial extent (field of view) of the image may be ascertained without having to hold any device up to the eye.
It is a further object of this invention to provide such a method of acquiring a picture electronically where the tonal characteristics of the picture may be ascertained without having to hold any device up to the eye.
It is a further object of this invention to provide such a method of acquiring a picture electronically where no apparent difference in body movement or gesture between when a picture is being taken and when no picture is being taken is detectable by others.
It is a further object of this invention to provide a photojournalist with a means of determining the composition of a picture from a display device that is located such that only the photojournalist can see the display device, and so that the photojour-nalist can ascertain the composition of a picture and take a picture or video and transmit these to one or more remote locations without the knowledge of others in the immediate environment.
It is a further object of this invention to provide a photojournalist with a means of collaborating with one or more remote entities, in matters pertaining to composition of a picture or video.
It is a further object of this invention to provide a photojournalist with a means of collaborating with one or more remote entities, in matters pertaining to an interview with one or more subjects.
It is a further object of this invention to provide the photojournalist with a means of determining the composition of the picture from a display device that is located such that only the photojournalist can see the display device, as well as a secondary additional display device that the photojournalist can show to others if and when the photojournalist desires to do so.
It is a further object of this invention to provide the photojournalist with a means of determining the composition of the picture from a display device that is located such that both the photojournalist as well as others can see it, if and when the photojournalist desires.
It is a further object of this invention to provide a wearable camera viewfinder means in which video is displayed on a viewfinder in such a way that all rays of light from the viewfinder that enter the eye appear to emanate from essentially the same direction as they would have had the apparatus not been worn.
It is a further object of this invention to provide a means for a user to experience additional information overlaid on top of his or her visual field of view such that the information is relevant to the imagery being viewed.
It is a further object of this invention to provide a means and apparatus for a user to capture a plurality of images of the same scene or objects, in a natural process of simply looking around, and then have these images combined together into a single image of increased spatial extent, spatial resolution, dynamic range, or tonal fidelity, either locally, or remotely, and if this processing is done locally, it is an object of this invention to facilitate wireless transmission of the finished image composite.
It is a further object of this invention to provide a viewfinder means in which the viewfinder has a focusing mechanism that is coupled to a focusing mechanism of a camera system, so that when the camera is focused on a particular object the viewfinder also presents that object in a manner such that when the apparatus moves relative to the user's eye, that the object appears to neither move with or against the movement of the eye, so that the rays of light entering the eye are approximately the same in direction as if the apparatus were not present.
It is a further object of this invention to provide a viewfinder means in which the viewfinder has a focusing mechanism that is coupled to a focusing mechanism of a camera system, so that when the camera is focused on a particular object the viewfinder also presents that object in the same focal depth plane as the object would appear to the user with the apparatus removed.
It is a further object of this invention to provide a viewfinder means in which the viewfinder has a focusing mechanism coupled with the focus of a camera, such that both can be operated remotely with a single control, by a technician or camera operator at a remote location.
It is a further object of this invention to provide a means by which a technician or camera operator at a remote location can signal to the wearer of the viewfinder which direction in which to turn, for best picture.
It is a further object of this invention to provide a means by which a technician or camera operator at a remote location can signal to the wearer of a wearable camera which has no local viewfinder, which direction in which to turn, for best picture, while the remote technician or camera operator remotely monitors the video signal from the wearable camera.
It is a further object of this invention to provide a viewfinder means in which the viewfinder has a focusing mechanism that is controlled by an automatic focusing mechanism of a camera system.
It is a further object of this invention to allow multiple photojournalists using the invention, at the same location, to collaborate in such a way that multiple camera viewpoints may be shared among the photojournalists so that they can advise each other on matters such as composition, or so that one or more experts at remote loca-tions can advise one or more of the photojournalists on matters such as composition or camera angle.
It is a further object of this invention to provide a new genre of journalism in which content may be produced and consumed using the same wearable camera and display system.
SUMMARY OF THE INVENTION: Informal review of what the new invention does One aspect of the invention allows a photojournalist to wear the apparatus con-tinuously and therefore always end up with the ability to produce a picture from something that was seen a couple of minutes ago.
Moreover, as an artistic tool for first-person perspective photojournalism, the apparatus allows the photojournalist to record, from a first-person-perspective, ex-periences that have been difficult to so record in the past. For example, a photo-journalist might be able to record the experience of looking through binoculars while riding horseback, or the experience of waterskiing, rope climbing, or the like. Such experiences captured from a first-person perspective provide a new genre of video by way of a wearable camera system with viewfinder means that goes beyond current state-of-the-art point of view sports videos (such as created by cameras mounted in sports helmets which have no viewfinder means.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention will now be described in more detail, by way of examples which in no way are meant to limit the scope of the invention, but, rather, these examples will serve to illustrate the invention with reference to the accompanying drawings, in which:
FIG. 1 is a diagram of a simple embodiment of the viewfinder portion of the invention in which a portion of the field of view of the journalist wearing the appratus is diverted into a camera by a mirror, and replaced with the view from a television screen.
FIG. 2 depicts the signal flow in which the camera transmits to a remote location, the image is processed, and the processed image is received by the apparatus and presented to the wearer upon the viewfinder means described above.
FIG. 3 is a block diagram of the new kind of computer system used in the inven-tion, in which the human user, in efFect, becomes part of the computational process.
FIG. 4 illustrates an embodiment of the covert data-entry portion of the invention in which a plurality of pushbutton switches are mounted to a belt.
FIG. 5 shows a typical application depicting three frames of the video sequence observed in the journalist's viewfinder, in which the journalist's perception of reality is altered by way of a simple ob j ect-tracking textual overlay on top of what would normally be seen through the viewfinder.
FIG. 6 shows an application in which the apparatus is used as a means of con-suming, rather than producing news, and in particular this diagram depicts a virtual newspaper as seen through the special eyeglasses.
While the invention shall now be described with reference to the preferred em-bodiments shown in the drawings, it should be understood that the intention is not to limit the invention only to the particular embodiments shown but rather to cover all alterations, modifications and equivalent arrangements possible within the scope of appended claims.
In a11 aspects of the present invention, references to "camera" mean any device or collection of devices capable of simultaneously determining a quantity of light arriving from a plurality of directions and or at a plurality of locations, or determining some other attribute of light arriving from a plurality of directions and or at a plurality of locations. Similarly references to "television" shall not be limited to just television monitors or traditional televisions used for the display of video from a camera near or distant, but shall also include computer data display means, computer data monitors, other video display devices, still picture display devices, ASCII text display devices, terminals, and the like.
Similarly, references to "photojournalist" shall mean all manner of journalist, pho-tographer, videographer, filmmaker, or the like.
DETAILED DESCRIPTION OF THE INVENTION WITH
REFERENCE TO DRAWINGS
Fig 1 shows an embodiment of the viewfinder portion of the system whereby rays of light spanning a visual angle from ray 1l0 to ray 120 enter the apparatus and are intercepted by a two-sided mirror 115, typically mounted at a 45 degree angle with respect to the optical axis of a camera 130. These rays of light enter camera 130. The video output of the camera 130 is displayed upon television screen possibly after having been processed on a body-worn computer system or the like.
A reflection of television screen 140 is seen in the other side of mirror 1l5, so that the television image of ray 110 appears as virtual ray 160 and the television image of ray l20 appears as ray 170. Since the camera 130 records an image image that is backwards, a backwards image is displayed on the television screen 140. Since the television l40 is observed in a mirror, the image is reversed again so that the view seen at eye location l90 is not backwards. In this way a portion of the wearer's visual field of view is replaced by the exact same subject matter, in perfect spatial register with the real world.
In practice, television 140 is a viewfinder with a focus adjustment, and the focus adjustment is driven by a servo mechanism controlled by an autofocus camera.
Thus camera 130 automatically focuses on the subject matter of interest, and controls the focus of viewfinder 130 so that the apparent distance to the object is the same while looking through the apparatus as with the apparatus removed.
It is desirable that embodiments of the wearable camera invention comprising man-ual focus cameras have the focus of the camera linked to the focus of the viewfinder so that both may be adjusted together with a single knob. Moreover, a camera with zoom lens may be used together with a viewfinder having zoom lens. The zoom mech-anisms are linked in such a way that the viewfinder image magnification is reduced as the camera magnification is increased. Through this appropriate linkage, any increase in magnification by the camera is negated exactly by decreasing the apparent size of the viewfinder image.
Focus may, alternatively, be controlled by a remote expert or technician, so that the journalist can concentrate on higher-level issues such as doing an interview.
The calibration of the autofocus zoom camera and the zoom viewfinder may be done by temporarily removing the mirror 115 and adjusting the focus and zoom of the viewfinder to maximize video feedback. This must be done for each zoom setting, so that the zoom of the viewfinder will properly track the zoom of the camera.
By using video feedback as a calibration tool, a computer system may monitor the video output of the camera while adjusting the viewfinder and generating a lookup table for the viewfinder settings corresponding to each camera setting. In this way, calibration may be automated during manufacture of the wearable camera system. Some similar embodiments of the wearable camera invention use two cameras and two viewfinders.
In some embodiments, the vergence of the viewfinders is linked to the focus mechanism of the viewfinders and the focus setting of cameras, so that there is a single automatic or manual focus adjustment for viewfinder vergence, camera vergence, viewfinder focus, and camera focus. Other embodiments of the wearable viewfinder system are described in patent application "WEARABLE CAMERA WITH VIEWFINDER
MEANS" .
Fig 2 depicts the system architecture in which camera 210 transmits through an-tenna 220 to remote site 230. At remote site 230, a collection of remote experts 240, some of which may be human experts (technicians, legal experts, scholars, etc.) and some of which may be computer agents or other kinds of computer programs. The experts at site 240 modify the images received from camera 210 and then send those images to transmitter 250 which is picked up by the journalist's receiver 270 and dis-played on the journalist's head mounted display (HMD) viewfinder 270. Camera and television (HMD) 270 are both typically housed in eyeglasses, and collectively, in the context of this entire system, allow the remote experts 240 to augment, diminish, or otherwise alter the journalist's visual perception of reality, and therefore establish not only a means by which images are transmitted from the journalist's camera to one or more remote sites, but also allow the remote sites to communicate directly with the journalist by way of modifying the image stream. Simple examples of this modification might include recognization of the identity of individuals in pictures sent by the journalist, and insertion of virtual name tags in the form of virtual text or graphics overlays upon the journalist's viewfinder 270.
Fig 3 depicts a personal imaging computer in which some of the functionality embodied in the panel of remote experts 240 is instead embodied in a wearable sys-tem. In actual practice, the expertise may come from computer programs running on systems worn on the body of the journalist, from systems remotely accessed by the journalist, or from remote human intelligence. Fig 3 is characterized by a com-puter that operates constantly 310, but is configured so that it does not restrict the journalist from doing other tasks. This unencumbering nature is expressed by signal flow 320 from the human, outward. Moreover, computer 3l0 is unmonopolizing of the journalist's attention, and is configured so that the journalist does not need to pay the level of attention to the computer that is typical of normal desktop or laptop computers. Thus the journalist uses a reality-interface metaphor to the computer, so that information, for example, might appear where the journalist is already looking (such as at the person he or she is interviewing), rather than requiring undivided attention of the journalist as might be the case with a conventional graphical user interface found on most desktop computers. This signal flow 330 denotes input from the environment to the human (journalist). There is also signal flow 340 from the computer to the journalist and signal flow from the journalist to the computer 350.
Since the computer 310 is constant in operation, at any time, it can respond to signal control 350, unlike a traditional laptop computer which must first be turned on and booted. Thus the system assists in being always ready to cover any newsworthy event that might happen without notice. Furthermore, since the computer 310 is always running, it is always taking in information 360 in the environment. For example, the system is always taking pictures and video whether or not there is a need for pictures or video. In this way, a retroactive record function may, for example, be implemented.
Moreover, a remote (or local) expert might recognize someone of importance even if the journalist is not paying attention to his or her viewfinder. There is a signal path 360 into the computer, and a signal path 370 out of the computer. By the combi-nation of 360 and 370, there is established a high-speed communications link.
The entire system is worn on the body of the journalist, within clothing 380, so that it is a personal effect of the journalist and is perceived by others as a personal effect of the journalist, rather than something that the journalist carries in a satchel or case.
In this way, it is less likely that others will ask the journalist to leave the apparatus behind (e.g. ask the journalist to leave his or her clothes behind).
Fig 4 illustrates a covert data-entry device. Spring-loaded rocker switches function as both momentary and latching switches, while pushbutton switches function as momentary-only switches. Belt mounted switches 410 and 420 are con-nected to the computer by connector 430. The belt may be concealed under a heavy sweater or jacket, and permits control of the body-worn computer system, such as may be desired to covertly type a query to a remote expert while conducting an in-vestigative interview. In this way, rather than having to call for remote advice by telephone, little or no sound is made in this process, and it is not as easily for others to eavesdrop on this virtual conversation, as might be facilitated by the UNIX
"talk"
command or the like.
Fig 5 depicts a computer-mediated reality as seen from the perspective of the jour-nalist. This figure depicts three frames of a video sequence taken from the journalist's viewfinder. The journalist, doing a story on the honesty of cashiers in explaining re-fund policies, looks at a cashier 510 in a department store. The viewfinder provides the journalist with additional information in the form of an overlay 520. As the jour-nalist turns his head to the right, the cashier 530 moves to the left in the viewfinder, as we normally expect when looking through a viewfinder. However, the extra in-formation 540 provided to the journalist moves with the cashier, so that it sustains the illusion of a rigid planar patch. In this way, the real and virtual objects remain connected. Even when the cashier 550 begins to fall off the left side of the viewfinder as when the journalist turns his head so far to the right as to no longer be able to see the cashier's face, the extra information 560 is still, in part, visible, and the illusion of attachment is sustained. This is an example of a reality-user interface that does not require that the journalist pay full attention to the computer task.
Fig 6 depicts an application in which the end-user uses the apparatus depicted in Fig 1 to read a newspaper. As the user walks down corridor 610, a newspaper is displayed on wall 620. The newspaper is presented to the reader in a way that it appears to be a rigid planar patch attached to the wall 620 of corridor 610.
In this way a journalist can write a story upon the walls of the corridor, and others who wear the appropriate apparatus can read the story. This may allow, for example, the journalist to write a story about this very corridor, and place it in the corridor for reading, so that those reading the story may, for example, re-live an experience as a virtual past.
BENEFITS OF THE INVENTION
The present invention allows the journalist to experience the camera over a long period of time. For example, after wearing the apparatus a11 day for several weeks, it begins to function as a true extension of the mind and body. In this way, pho-tographic composition is much more optimal, because the act of taking pictures or shooting video no longer requires conscious thought or effort. Moreover, the inten-tionality of the picture-taking process is not evident to others, because picture-taking is not preceeded by a gesture such as holding a viewfinder object up to the eye. The wearable viewfinder is an important element of the wearable camera invention al-lowing the j ournalist to experience everyday life through a screen, and therefore be always ready to capture anything that might happen, or even anything that might have happened previously by virtue of the retroactive record capability of the inven-tion. Moreover, additional information beyond just exposure and shutter speed may be displayed in the camera viewfinder. For example, the camera allows the journal-ist to augment, diminish, or otherwise alter his or her perception of visual reality.
This mediated-reality experience may be shared. The journalist may allow others to alter his or her perception of reality. In this way the invention is useful as a new communications medium, in the context of collaborative photojournalism, collabo-rative videography, and telepresence for intelligence augmentation for purposes of conducting an interview.
OTHER EMBODIMENTS
From the foregoing description, it will thus be evident that the present invention provides a design for a wearable camera with viewfinder means. As various changes can be made in the above embodiments and operating methods without departing from the spirit or scope of the following claims, it is intended that a11 matter contained in the above description or shown in the accompanying drawings should be interpreted as illustrative and not in a limiting sense.
Variations or modifications to the design and construction of this invention, within the scope of the appended claims, may occur to those skilled in the art upon reviewing the disclosure herein. Such variations or modifications, if within the spirit of this invention, are intended to be encompassed within the scope of any claims to patent protection issuing upon this invention.
For such compositional and interactional capabilities, a simple air-based viewfinder is inadequate.
Wearable display devices have been described, such as in U.S. Pat. No.5546099, U.S. Pat. No.5708449, U.S. Pat. No.5331333, U.S. Pat. No.4636866, and may be used as a starting point for implementing the proposed invention, through the addition of the appropriate light sensing apparatus and wireless communications means.
U.S.
Pat. No.5640221 also proposes an eye-tracking device which may be used in the context of the proposed invention.
DESCRIPTION OF THE INVENTION
The proposed invention facilitates a new form of visual art, in which the artist may capture, with relatively little effort, a visual experience as viewed from his or her own perspective. With some practice, it is possible to develop a very steady body posture and mode of movement that best produces video of the genre pertaining to this invention. Because the apparatus is lightweight and close to the head, there is not the protrusion associated with carrying a hand-held camera. Also because components of the proposed invention are mounted very close to the head, in a manner that balances the weight distribution as well as minimizes the moment of inertia about the rotational axis of the neck, the head can be turned quickly while wearing the apparatus.
This allows one to record the experiences of ordinary day-to-day activities from a first-person perspective, and, because of the proposed communications infrastructure, to convey these experiences to a remote entity. Moreover, because both hands are free, much better balance and posture is possible while using the apparatus. Anyone skilled in the arts of body movement control as is learned in the martial arts such as karate, as well as in dance, most notably ballet, will have little difficulty capturing exceptionally high quality video using the proposed invention.
With the prior art, the best camera operators tend to be very large people who have trained for many years in the art of smooth control of the cumbersome video or motion picture film cameras used. In addition to requiring a very large person to opti-mally operate such cameras, various stabilization devices are often used, which make the apparatus even more cumbersome. The proposed wearable camera invention may be optimally operated by people of any size. Even young children can become quite proficient in the use of the proposed invention, and could make excellent photojour-nalists, especially if given the capabilities of a network of adult experts.
A typical embodiment of the invention comprises one or two spatial light mod-ulators or other display means built into a pair of eyeglasses together with one or more light sensor arrays, a body-worn computer and image processing system, and a high-speed wireless communications system. Typically one or more CCD (charge coupled device image sensor arrays and appropriate optical elements comprise the camera portion of the invention. Typically a beamsplitter or a mirror silvered on both sides is used to combine the image of the viewfinder with the apparent position of the camera. The viewfinder is not j ust a means of determining the extent of coverage of the camera in a natural manner, but it is also a communications means, so that, for example, a remote expert may provide advice in the form of text superimposed inside the viewfinder. In this manner the viewfinder functions as a teleprompter, as well as a means for j udging photographic or video composition. Moreover, one or more remote experts may superimpose graphical elements in the viewfinder, such as a cursor or arrow that points to an object as depicted in the viewfinder.
In this way the wearer may collaborate with one or more remote experts on matters such as scene content, or, for example, specific details of a photographic composition. Fi-nally, the collaboration may take the form of computer graphics rendering inserted into the viewfinder, together with the images of real objects. This allows the user to experience a computer-mediated reality in which there is a much more rich form of collaborative potential.
SUMMARY OF THE INVENTION: OBJECTS AND ADVANTAGES
It is an object of this invention to provide a method of positioning a camera in which both hands are left free.
It is a further object of this invention to provide a means of acquiring a picture electronically where the spatial extent (field of view) of the image may be ascertained without having to hold any device up to the eye.
It is a further object of this invention to provide such a method of acquiring a picture electronically where the tonal characteristics of the picture may be ascertained without having to hold any device up to the eye.
It is a further object of this invention to provide such a method of acquiring a picture electronically where no apparent difference in body movement or gesture between when a picture is being taken and when no picture is being taken is detectable by others.
It is a further object of this invention to provide a photojournalist with a means of determining the composition of a picture from a display device that is located such that only the photojournalist can see the display device, and so that the photojour-nalist can ascertain the composition of a picture and take a picture or video and transmit these to one or more remote locations without the knowledge of others in the immediate environment.
It is a further object of this invention to provide a photojournalist with a means of collaborating with one or more remote entities, in matters pertaining to composition of a picture or video.
It is a further object of this invention to provide a photojournalist with a means of collaborating with one or more remote entities, in matters pertaining to an interview with one or more subjects.
It is a further object of this invention to provide the photojournalist with a means of determining the composition of the picture from a display device that is located such that only the photojournalist can see the display device, as well as a secondary additional display device that the photojournalist can show to others if and when the photojournalist desires to do so.
It is a further object of this invention to provide the photojournalist with a means of determining the composition of the picture from a display device that is located such that both the photojournalist as well as others can see it, if and when the photojournalist desires.
It is a further object of this invention to provide a wearable camera viewfinder means in which video is displayed on a viewfinder in such a way that all rays of light from the viewfinder that enter the eye appear to emanate from essentially the same direction as they would have had the apparatus not been worn.
It is a further object of this invention to provide a means for a user to experience additional information overlaid on top of his or her visual field of view such that the information is relevant to the imagery being viewed.
It is a further object of this invention to provide a means and apparatus for a user to capture a plurality of images of the same scene or objects, in a natural process of simply looking around, and then have these images combined together into a single image of increased spatial extent, spatial resolution, dynamic range, or tonal fidelity, either locally, or remotely, and if this processing is done locally, it is an object of this invention to facilitate wireless transmission of the finished image composite.
It is a further object of this invention to provide a viewfinder means in which the viewfinder has a focusing mechanism that is coupled to a focusing mechanism of a camera system, so that when the camera is focused on a particular object the viewfinder also presents that object in a manner such that when the apparatus moves relative to the user's eye, that the object appears to neither move with or against the movement of the eye, so that the rays of light entering the eye are approximately the same in direction as if the apparatus were not present.
It is a further object of this invention to provide a viewfinder means in which the viewfinder has a focusing mechanism that is coupled to a focusing mechanism of a camera system, so that when the camera is focused on a particular object the viewfinder also presents that object in the same focal depth plane as the object would appear to the user with the apparatus removed.
It is a further object of this invention to provide a viewfinder means in which the viewfinder has a focusing mechanism coupled with the focus of a camera, such that both can be operated remotely with a single control, by a technician or camera operator at a remote location.
It is a further object of this invention to provide a means by which a technician or camera operator at a remote location can signal to the wearer of the viewfinder which direction in which to turn, for best picture.
It is a further object of this invention to provide a means by which a technician or camera operator at a remote location can signal to the wearer of a wearable camera which has no local viewfinder, which direction in which to turn, for best picture, while the remote technician or camera operator remotely monitors the video signal from the wearable camera.
It is a further object of this invention to provide a viewfinder means in which the viewfinder has a focusing mechanism that is controlled by an automatic focusing mechanism of a camera system.
It is a further object of this invention to allow multiple photojournalists using the invention, at the same location, to collaborate in such a way that multiple camera viewpoints may be shared among the photojournalists so that they can advise each other on matters such as composition, or so that one or more experts at remote loca-tions can advise one or more of the photojournalists on matters such as composition or camera angle.
It is a further object of this invention to provide a new genre of journalism in which content may be produced and consumed using the same wearable camera and display system.
SUMMARY OF THE INVENTION: Informal review of what the new invention does One aspect of the invention allows a photojournalist to wear the apparatus con-tinuously and therefore always end up with the ability to produce a picture from something that was seen a couple of minutes ago.
Moreover, as an artistic tool for first-person perspective photojournalism, the apparatus allows the photojournalist to record, from a first-person-perspective, ex-periences that have been difficult to so record in the past. For example, a photo-journalist might be able to record the experience of looking through binoculars while riding horseback, or the experience of waterskiing, rope climbing, or the like. Such experiences captured from a first-person perspective provide a new genre of video by way of a wearable camera system with viewfinder means that goes beyond current state-of-the-art point of view sports videos (such as created by cameras mounted in sports helmets which have no viewfinder means.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention will now be described in more detail, by way of examples which in no way are meant to limit the scope of the invention, but, rather, these examples will serve to illustrate the invention with reference to the accompanying drawings, in which:
FIG. 1 is a diagram of a simple embodiment of the viewfinder portion of the invention in which a portion of the field of view of the journalist wearing the appratus is diverted into a camera by a mirror, and replaced with the view from a television screen.
FIG. 2 depicts the signal flow in which the camera transmits to a remote location, the image is processed, and the processed image is received by the apparatus and presented to the wearer upon the viewfinder means described above.
FIG. 3 is a block diagram of the new kind of computer system used in the inven-tion, in which the human user, in efFect, becomes part of the computational process.
FIG. 4 illustrates an embodiment of the covert data-entry portion of the invention in which a plurality of pushbutton switches are mounted to a belt.
FIG. 5 shows a typical application depicting three frames of the video sequence observed in the journalist's viewfinder, in which the journalist's perception of reality is altered by way of a simple ob j ect-tracking textual overlay on top of what would normally be seen through the viewfinder.
FIG. 6 shows an application in which the apparatus is used as a means of con-suming, rather than producing news, and in particular this diagram depicts a virtual newspaper as seen through the special eyeglasses.
While the invention shall now be described with reference to the preferred em-bodiments shown in the drawings, it should be understood that the intention is not to limit the invention only to the particular embodiments shown but rather to cover all alterations, modifications and equivalent arrangements possible within the scope of appended claims.
In a11 aspects of the present invention, references to "camera" mean any device or collection of devices capable of simultaneously determining a quantity of light arriving from a plurality of directions and or at a plurality of locations, or determining some other attribute of light arriving from a plurality of directions and or at a plurality of locations. Similarly references to "television" shall not be limited to just television monitors or traditional televisions used for the display of video from a camera near or distant, but shall also include computer data display means, computer data monitors, other video display devices, still picture display devices, ASCII text display devices, terminals, and the like.
Similarly, references to "photojournalist" shall mean all manner of journalist, pho-tographer, videographer, filmmaker, or the like.
DETAILED DESCRIPTION OF THE INVENTION WITH
REFERENCE TO DRAWINGS
Fig 1 shows an embodiment of the viewfinder portion of the system whereby rays of light spanning a visual angle from ray 1l0 to ray 120 enter the apparatus and are intercepted by a two-sided mirror 115, typically mounted at a 45 degree angle with respect to the optical axis of a camera 130. These rays of light enter camera 130. The video output of the camera 130 is displayed upon television screen possibly after having been processed on a body-worn computer system or the like.
A reflection of television screen 140 is seen in the other side of mirror 1l5, so that the television image of ray 110 appears as virtual ray 160 and the television image of ray l20 appears as ray 170. Since the camera 130 records an image image that is backwards, a backwards image is displayed on the television screen 140. Since the television l40 is observed in a mirror, the image is reversed again so that the view seen at eye location l90 is not backwards. In this way a portion of the wearer's visual field of view is replaced by the exact same subject matter, in perfect spatial register with the real world.
In practice, television 140 is a viewfinder with a focus adjustment, and the focus adjustment is driven by a servo mechanism controlled by an autofocus camera.
Thus camera 130 automatically focuses on the subject matter of interest, and controls the focus of viewfinder 130 so that the apparent distance to the object is the same while looking through the apparatus as with the apparatus removed.
It is desirable that embodiments of the wearable camera invention comprising man-ual focus cameras have the focus of the camera linked to the focus of the viewfinder so that both may be adjusted together with a single knob. Moreover, a camera with zoom lens may be used together with a viewfinder having zoom lens. The zoom mech-anisms are linked in such a way that the viewfinder image magnification is reduced as the camera magnification is increased. Through this appropriate linkage, any increase in magnification by the camera is negated exactly by decreasing the apparent size of the viewfinder image.
Focus may, alternatively, be controlled by a remote expert or technician, so that the journalist can concentrate on higher-level issues such as doing an interview.
The calibration of the autofocus zoom camera and the zoom viewfinder may be done by temporarily removing the mirror 115 and adjusting the focus and zoom of the viewfinder to maximize video feedback. This must be done for each zoom setting, so that the zoom of the viewfinder will properly track the zoom of the camera.
By using video feedback as a calibration tool, a computer system may monitor the video output of the camera while adjusting the viewfinder and generating a lookup table for the viewfinder settings corresponding to each camera setting. In this way, calibration may be automated during manufacture of the wearable camera system. Some similar embodiments of the wearable camera invention use two cameras and two viewfinders.
In some embodiments, the vergence of the viewfinders is linked to the focus mechanism of the viewfinders and the focus setting of cameras, so that there is a single automatic or manual focus adjustment for viewfinder vergence, camera vergence, viewfinder focus, and camera focus. Other embodiments of the wearable viewfinder system are described in patent application "WEARABLE CAMERA WITH VIEWFINDER
MEANS" .
Fig 2 depicts the system architecture in which camera 210 transmits through an-tenna 220 to remote site 230. At remote site 230, a collection of remote experts 240, some of which may be human experts (technicians, legal experts, scholars, etc.) and some of which may be computer agents or other kinds of computer programs. The experts at site 240 modify the images received from camera 210 and then send those images to transmitter 250 which is picked up by the journalist's receiver 270 and dis-played on the journalist's head mounted display (HMD) viewfinder 270. Camera and television (HMD) 270 are both typically housed in eyeglasses, and collectively, in the context of this entire system, allow the remote experts 240 to augment, diminish, or otherwise alter the journalist's visual perception of reality, and therefore establish not only a means by which images are transmitted from the journalist's camera to one or more remote sites, but also allow the remote sites to communicate directly with the journalist by way of modifying the image stream. Simple examples of this modification might include recognization of the identity of individuals in pictures sent by the journalist, and insertion of virtual name tags in the form of virtual text or graphics overlays upon the journalist's viewfinder 270.
Fig 3 depicts a personal imaging computer in which some of the functionality embodied in the panel of remote experts 240 is instead embodied in a wearable sys-tem. In actual practice, the expertise may come from computer programs running on systems worn on the body of the journalist, from systems remotely accessed by the journalist, or from remote human intelligence. Fig 3 is characterized by a com-puter that operates constantly 310, but is configured so that it does not restrict the journalist from doing other tasks. This unencumbering nature is expressed by signal flow 320 from the human, outward. Moreover, computer 3l0 is unmonopolizing of the journalist's attention, and is configured so that the journalist does not need to pay the level of attention to the computer that is typical of normal desktop or laptop computers. Thus the journalist uses a reality-interface metaphor to the computer, so that information, for example, might appear where the journalist is already looking (such as at the person he or she is interviewing), rather than requiring undivided attention of the journalist as might be the case with a conventional graphical user interface found on most desktop computers. This signal flow 330 denotes input from the environment to the human (journalist). There is also signal flow 340 from the computer to the journalist and signal flow from the journalist to the computer 350.
Since the computer 310 is constant in operation, at any time, it can respond to signal control 350, unlike a traditional laptop computer which must first be turned on and booted. Thus the system assists in being always ready to cover any newsworthy event that might happen without notice. Furthermore, since the computer 310 is always running, it is always taking in information 360 in the environment. For example, the system is always taking pictures and video whether or not there is a need for pictures or video. In this way, a retroactive record function may, for example, be implemented.
Moreover, a remote (or local) expert might recognize someone of importance even if the journalist is not paying attention to his or her viewfinder. There is a signal path 360 into the computer, and a signal path 370 out of the computer. By the combi-nation of 360 and 370, there is established a high-speed communications link.
The entire system is worn on the body of the journalist, within clothing 380, so that it is a personal effect of the journalist and is perceived by others as a personal effect of the journalist, rather than something that the journalist carries in a satchel or case.
In this way, it is less likely that others will ask the journalist to leave the apparatus behind (e.g. ask the journalist to leave his or her clothes behind).
Fig 4 illustrates a covert data-entry device. Spring-loaded rocker switches function as both momentary and latching switches, while pushbutton switches function as momentary-only switches. Belt mounted switches 410 and 420 are con-nected to the computer by connector 430. The belt may be concealed under a heavy sweater or jacket, and permits control of the body-worn computer system, such as may be desired to covertly type a query to a remote expert while conducting an in-vestigative interview. In this way, rather than having to call for remote advice by telephone, little or no sound is made in this process, and it is not as easily for others to eavesdrop on this virtual conversation, as might be facilitated by the UNIX
"talk"
command or the like.
Fig 5 depicts a computer-mediated reality as seen from the perspective of the jour-nalist. This figure depicts three frames of a video sequence taken from the journalist's viewfinder. The journalist, doing a story on the honesty of cashiers in explaining re-fund policies, looks at a cashier 510 in a department store. The viewfinder provides the journalist with additional information in the form of an overlay 520. As the jour-nalist turns his head to the right, the cashier 530 moves to the left in the viewfinder, as we normally expect when looking through a viewfinder. However, the extra in-formation 540 provided to the journalist moves with the cashier, so that it sustains the illusion of a rigid planar patch. In this way, the real and virtual objects remain connected. Even when the cashier 550 begins to fall off the left side of the viewfinder as when the journalist turns his head so far to the right as to no longer be able to see the cashier's face, the extra information 560 is still, in part, visible, and the illusion of attachment is sustained. This is an example of a reality-user interface that does not require that the journalist pay full attention to the computer task.
Fig 6 depicts an application in which the end-user uses the apparatus depicted in Fig 1 to read a newspaper. As the user walks down corridor 610, a newspaper is displayed on wall 620. The newspaper is presented to the reader in a way that it appears to be a rigid planar patch attached to the wall 620 of corridor 610.
In this way a journalist can write a story upon the walls of the corridor, and others who wear the appropriate apparatus can read the story. This may allow, for example, the journalist to write a story about this very corridor, and place it in the corridor for reading, so that those reading the story may, for example, re-live an experience as a virtual past.
BENEFITS OF THE INVENTION
The present invention allows the journalist to experience the camera over a long period of time. For example, after wearing the apparatus a11 day for several weeks, it begins to function as a true extension of the mind and body. In this way, pho-tographic composition is much more optimal, because the act of taking pictures or shooting video no longer requires conscious thought or effort. Moreover, the inten-tionality of the picture-taking process is not evident to others, because picture-taking is not preceeded by a gesture such as holding a viewfinder object up to the eye. The wearable viewfinder is an important element of the wearable camera invention al-lowing the j ournalist to experience everyday life through a screen, and therefore be always ready to capture anything that might happen, or even anything that might have happened previously by virtue of the retroactive record capability of the inven-tion. Moreover, additional information beyond just exposure and shutter speed may be displayed in the camera viewfinder. For example, the camera allows the journal-ist to augment, diminish, or otherwise alter his or her perception of visual reality.
This mediated-reality experience may be shared. The journalist may allow others to alter his or her perception of reality. In this way the invention is useful as a new communications medium, in the context of collaborative photojournalism, collabo-rative videography, and telepresence for intelligence augmentation for purposes of conducting an interview.
OTHER EMBODIMENTS
From the foregoing description, it will thus be evident that the present invention provides a design for a wearable camera with viewfinder means. As various changes can be made in the above embodiments and operating methods without departing from the spirit or scope of the following claims, it is intended that a11 matter contained in the above description or shown in the accompanying drawings should be interpreted as illustrative and not in a limiting sense.
Variations or modifications to the design and construction of this invention, within the scope of the appended claims, may occur to those skilled in the art upon reviewing the disclosure herein. Such variations or modifications, if within the spirit of this invention, are intended to be encompassed within the scope of any claims to patent protection issuing upon this invention.
Claims (22)
1. A body-worn electronic display means capable of display of a picture, video, or graphic information, where said display means is affixed to the body in such a manner as to leave both of the wearer's hands free, and where said display means is capable of the display of information from a body worn computer system, and where said computer system is capable of receiving input from a body-worn electronic camera.
2. Means and apparatus as described in Claim 1 comprising a wireless communications link to one or more remote sites.
3. Means and apparatus as described in Claim 1 comprising a wireless communications link to a one or more remote computer networks.
4. Means and apparatus as described in Claim 2 where said communications link is a video communications link.
5. Means and apparatus as described in Claim 4 where said video communications link permits images to be sent from said camera to said one or more remote sites.
6. Means and apparatus as described in Claim 4 where said video communications link permits images to be simultaneously sent from said camera to said one or more remote sites and also permits images to be sent from said one or more remote sites to said means capable of display of a picture, video, or graphic information.
7. Means and apparatus as described in Claim 4 where said video communications link permits images to be simultaneously sent from said camera to said one or more remote sites, modified at said one or more remote sites, also then sent from said one or more remote sites to said means display means.
8. Means and apparatus as described in Claim 2 where a separate camera and camera viewfinder are mounted to a helmet, and said camera is connected to a body-worn computer system in such a way that said computer system may capture images from said camera, and where said viewfinder comprises a computer monitor connected to said computer system.
9. Means and apparatus as described in Claim 4 where a separate camera and camera viewfinder are mounted to a helmet, and said camera is connected to a body-worn computer system in such a way that said computer system may capture images from said camera, and where said viewfinder comprises a computer monitor connected to said computer system.
10. Means and apparatus as described in Claim 3 where a separate camera and camera viewfinder are mounted to a helmet, and said camera is connected to a body-worn computer system in such a way that said computer system may capture images from said camera, and where said viewfinder comprises a computer monitor connected to said computer system.
11. Means and apparatus as described in Claim 2 where a separate camera and camera computer monitor are mounted to a pair of eyeglasses, and said camera is connected to a body-worn computer system in such a way that said computer system may capture images from said camera, and where said computer system may display said images upon said computer monitor.
12. Means and apparatus as described in Claim 4 where a separate camera and camera computer monitor are mounted to a pair of eyeglasses, and said camera is connected to a body-worn computer system in such a way that said computer system may capture images from said camera, and where said computer system may display said images upon said computer monitor.
13. Means and apparatus as described in Claim 3 where a separate camera and camera computer monitor are mounted to a pair of eyeglasses, and said camera is connected to a body-worn computer system in such a way that said computer system may capture images from said camera, and where said computer system may display said images upon said computer monitor.
14. Means and apparatus as described in Claim 2 where some of the rays of light that would enter the eye in the absence of said means and apparatus are diverted to said camera by said means and apparatus and displayed in such a manner that each rays of light from said display device enters the eye in such a way that it is essentially collinear to the corresponding ray of light that initially entered said means and apparatus.
15. Means and apparatus as described in Claim 4 where some of the rays of light that would enter the eye in the absence of said means and apparatus are diverted to said camera by said means and apparatus and displayed in such a manner that each rays of light from said display device enters the eye in such a way that it is essentially collinear to the corresponding ray of light that initially entered said means and apparatus.
16. Means and apparatus as described in Claim 3 where some of the rays of light that would enter the eye in the absence of said means and apparatus are diverted to said camera by said means and apparatus and displayed in such a manner that each rays of light from said display device enters the eye in such a way that it is essentially collinear to the corresponding ray of light that initially entered said means and apparatus.
17. Means and apparatus as described in Claim 2 where a portion of each ray of light that would enter the eye in the absence of said means and apparatus is diverted to said camera by said means and apparatus and displayed in such a manner that said portion of each ray of light from said display device enters the eye in such a way that when the signal from said camera is displayed upon said display device, the rays of light entering the eye each correspond to subject matter that is essentially equivalent to that which would have been observed in the absence of said means and apparatus.
18. Means and apparatus as described in Claim 4 where a portion of each ray of light that would enter the eye in the absence of said means and apparatus is diverted to said camera by said means and apparatus and displayed in such a manner that said portion of each ray of light from said display device enters the eye in such a way that when the signal from said camera is displayed upon said display device, the rays of light entering the eye each correspond to subject matter that is essentially equivalent to that which would have been observed in the absence of said means and apparatus.
19. Means and apparatus as described in Claim 3 where a portion of each ray of light that would enter the eye in the absence of said means and apparatus is diverted to said camera by said means and apparatus and displayed in such a manner that said portion of each ray of light from said display device enters the eye in such a way that when the signal from said camera is displayed upon said display device, the rays of light entering the eye each correspond to subject matter that is essentially equivalent to that which would have been observed in the absence of said means and apparatus.
20. Means and apparatus as described in Claim 17 where said camera and said display are mounted within a pair of eyeglasses.
21. Means and apparatus as described in Claim 18 where said camera and said display are mounted within a pair of eyeglasses.
22. Means and apparatus as described in Claim 19 where said camera and said display are mounted within a pair of eyeglasses.
Priority Applications (14)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA002235030A CA2235030A1 (en) | 1998-02-02 | 1998-04-14 | System for electronic newsgathering, documentary video, and photojournalism |
| CA002256918A CA2256918C (en) | 1998-02-02 | 1998-12-24 | Aremac--based means and apparatus for interaction with computer, or one or more other people, through a camera |
| CA002256920A CA2256920A1 (en) | 1998-02-02 | 1998-12-29 | Lenstop camera viewfinder or computer data display having appearance of ordinary reading glasses or half glasses |
| CA002256922A CA2256922C (en) | 1998-02-02 | 1998-12-31 | Aiming and compositional means for head--worn camera |
| CA002261376A CA2261376A1 (en) | 1998-02-02 | 1999-02-01 | Means and apparatus for acquiring, processing, and combining multiple exposures of the same scene or objects to different illuminations |
| CA002264973A CA2264973A1 (en) | 1998-03-15 | 1999-03-15 | Eye-tap for electronic newsgathering, documentary video, photojournalism, and personal safety |
| JP2000538502A JP2002508635A (en) | 1998-03-25 | 1999-03-25 | Eye tap camera used in electronic news gathering, documentary video production, photojournalism, and for life and body safety |
| AT99910052T ATE241245T1 (en) | 1998-03-25 | 1999-03-25 | VISOR FOR ELECTRONIC NEWS COLLECTION, DOCUMENTARY VIDEO, PHOTOJOURNALISM AND PERSONAL SECURITY |
| EP99910052A EP1066717B1 (en) | 1998-03-25 | 1999-03-25 | Eye-tap for electronic newsgathering, documentary video, photojournalism, and personal safety |
| DE69908108T DE69908108D1 (en) | 1998-03-25 | 1999-03-25 | VISOR FOR ELECTRONIC MESSAGE COLLECTION, DOCUMENTARY VIDEO, PHOTOJOURNALISM AND PERSONAL SECURITY |
| AU29174/99A AU2917499A (en) | 1998-03-25 | 1999-03-25 | Eye-tap for electronic newsgathering, documentary video, photojournalism, and personal safety |
| PCT/CA1999/000248 WO1999049655A1 (en) | 1998-03-25 | 1999-03-25 | Eye-tap for electronic newsgathering, documentary video, photojournalism, and personal safety |
| US09/275,775 US6614408B1 (en) | 1998-03-25 | 1999-03-25 | Eye-tap for electronic newsgathering, documentary video, photojournalism, and personal safety |
| CA002267877A CA2267877A1 (en) | 1998-04-14 | 1999-04-01 | Means and apparatus for collegial identification of persons such as officials asking for identification |
Applications Claiming Priority (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA2228403 | 1998-02-02 | ||
| CA002228403A CA2228403A1 (en) | 1998-02-02 | 1998-02-02 | Means and apparatus for aquiring, processing, and combining multiple exposures of the same scene or objects to different illuminations |
| CA002233047A CA2233047C (en) | 1998-02-02 | 1998-03-25 | Wearable camera system with viewfinder means |
| CA2233047 | 1998-03-25 | ||
| CA002235030A CA2235030A1 (en) | 1998-02-02 | 1998-04-14 | System for electronic newsgathering, documentary video, and photojournalism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2235030A1 true CA2235030A1 (en) | 1999-08-02 |
Family
ID=29407066
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002235030A Abandoned CA2235030A1 (en) | 1998-02-02 | 1998-04-14 | System for electronic newsgathering, documentary video, and photojournalism |
Country Status (1)
| Country | Link |
|---|---|
| CA (1) | CA2235030A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11392636B2 (en) | 2013-10-17 | 2022-07-19 | Nant Holdings Ip, Llc | Augmented reality position-based service, methods, and systems |
| US11854153B2 (en) | 2011-04-08 | 2023-12-26 | Nant Holdings Ip, Llc | Interference based augmented reality hosting platforms |
| US12118581B2 (en) | 2011-11-21 | 2024-10-15 | Nant Holdings Ip, Llc | Location-based transaction fraud mitigation methods and systems |
-
1998
- 1998-04-14 CA CA002235030A patent/CA2235030A1/en not_active Abandoned
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11854153B2 (en) | 2011-04-08 | 2023-12-26 | Nant Holdings Ip, Llc | Interference based augmented reality hosting platforms |
| US11869160B2 (en) | 2011-04-08 | 2024-01-09 | Nant Holdings Ip, Llc | Interference based augmented reality hosting platforms |
| US11967034B2 (en) | 2011-04-08 | 2024-04-23 | Nant Holdings Ip, Llc | Augmented reality object management system |
| US12182953B2 (en) | 2011-04-08 | 2024-12-31 | Nant Holdings Ip, Llc | Augmented reality object management system |
| US12118581B2 (en) | 2011-11-21 | 2024-10-15 | Nant Holdings Ip, Llc | Location-based transaction fraud mitigation methods and systems |
| US11392636B2 (en) | 2013-10-17 | 2022-07-19 | Nant Holdings Ip, Llc | Augmented reality position-based service, methods, and systems |
| US12008719B2 (en) | 2013-10-17 | 2024-06-11 | Nant Holdings Ip, Llc | Wide area augmented reality location-based services |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6614408B1 (en) | Eye-tap for electronic newsgathering, documentary video, photojournalism, and personal safety | |
| CA2233047C (en) | Wearable camera system with viewfinder means | |
| EP1064783B1 (en) | Wearable camera system with viewfinder means | |
| US20020085843A1 (en) | Wearable camera system with viewfinder means | |
| Mann | 'WearCam'(The wearable camera): personal imaging systems for long-term use in wearable tetherless computer-mediated reality and personal photo/videographic memory prosthesis | |
| US20020030637A1 (en) | Aremac-based means and apparatus for interaction with computer, or one or more other people, through a camera | |
| Mann | “Smart clothing” wearable multimedia computing and “personal imaging” to restore the technological balance between people and their environments | |
| CN101365061B (en) | Imaging apparatus, imaging system, and imaging method | |
| US20210281802A1 (en) | IMPROVED METHOD AND SYSTEM FOR VIDEO CONFERENCES WITH HMDs | |
| CN113655887A (en) | Virtual reality equipment and static screen recording method | |
| WO2000020951A2 (en) | Hand held computer with see-through display | |
| RU2157054C2 (en) | Method for production of video programs and device which implements said method | |
| CA2237939C (en) | Personal imaging system with viewfinder and annotation means | |
| JPH05303053A (en) | Head mount display device | |
| EP1066717B1 (en) | Eye-tap for electronic newsgathering, documentary video, photojournalism, and personal safety | |
| CA2235030A1 (en) | System for electronic newsgathering, documentary video, and photojournalism | |
| GB2292038A (en) | Video camera with head mounted display | |
| EP1644910A2 (en) | Methods and apparatuses for managing and presenting content through a spherical display device | |
| CA2248473C (en) | Eyetap camera or partial reality mediator having appearance of ordinary eyeglasses | |
| CA2247649C (en) | Covert camera viewfinder or display having appearance of ordinary eyeglasses | |
| CA2246697A1 (en) | Reality mediator with viewfinder means | |
| CA2264973A1 (en) | Eye-tap for electronic newsgathering, documentary video, photojournalism, and personal safety | |
| EP1664850A2 (en) | Methods and apparatuses for capturing and viewing content through a multi-lens device | |
| CA2256918C (en) | Aremac--based means and apparatus for interaction with computer, or one or more other people, through a camera | |
| CA2256920A1 (en) | Lenstop camera viewfinder or computer data display having appearance of ordinary reading glasses or half glasses |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FZDE | Discontinued |