Video mapping/with webcams

It has been proposed that this page or section be merged with video mapping. (Discuss)

Video mapping with webcams

• 

  left QC 500

• 

  front QC 960

• 

  right QC300

In April 2010, some experiments with three HD-WebCams in a car were started. Goals were good video quality with low-budget equipment. Here is a dirty "braindump". Needs some clean up over the time.

Why using WebCams and a laptop computer?

• WebCams are relatively cheap
• Hard Disk space is also cheap
• Sufficient memory of hard disk
• (Practically) unlimited file systems and video containers available
• many osm users alreasy own a laptop; netbook still cheaper than a professional camera
• unique system time for GPS log and videos should make synchronisation easy
• no problem with interlacing

What is possible?

Street name signs, street signs, lanes, house numbers, nearly any POIs

What are the disadvantages?

• Car is precondition for this equipment at this time because of energy reasons (consumes about 40 Watts). (Bicycle could get a huge battery)
• Cars cannot get everywhere.
• Webcams normally have a low-cost CMOS-sensor, whitch features a weak dynamic, meaning bright and dark parts of the picture getting lost rather rapid.
• The exposure driven by firmware is optimized for in-house usage. Wide parts of bright sky in the picture result in underexposure of object of interest. Idea: Client-side exposure regulation for better results. Should feature:
• Optimized for short exposure times in general by using a higher gain (→ pay with noise), to reduce motion blur (thats only necessary on not-ideal weather conditions);
• Use only down half of picture to determine exposure to get most information of POIs(→ pay with overexposured/outshining sky);
• Faster regulation algorithm (→ doesn't need to look nice in running video, but get out most information out of picture)

Answer to mentioned Problems

Problems:

   * The video will be rather boring to watch!

- not true for if you are glad about information. In urban areas, nearly every single frame contains mappable stuff (house numbers). On Motorroads, there is still a lot to do with limits, lanes, ... (Traffic sign recognition shouldn't be too complicated).

   * It would take as long as the car journey itself, except that you could fast-forward some bits.

- The truth is, it definitely takes much longer (maybe quad times) because of the high information density. But if you don't have fun to fill spare time doing that work, let it be.

   * If you can't see a street sign on the video, then you've failed to map that street. [...] More cameras??

- yes, use at least three cameras, problem mostly solved. The risk of information loss is not zero, but much less than all other known mapping techniques.

   * Video storage space (e.g. tape length) might be a limiting factor. 

- solved by hard disk with this project.

Hardware

Logitech C300

• Properties: 1.3 Mpixel, plastic lens, manual focus (lens adjustable by winding), max. resolution: 1280x1024 @ 15 fps (Hardware MJPEG)
• Price: about 35€
• Subjective: Suitable as frontview, satisfying as sideview. Readability of (german) street name signs: <10m distance. Side house numbers: satisfying.
• Best choice for the money

Logitech C500

• Product properties: 1.3Mpixel, glass lens, no focus (Fixfocus with good depth of sharpness), max. resolution 1280x1024 @ 15 fps (Hardware MJPEG)
• Price: about 45€
• Subjective: Suitable as frontview, satisfying as sideview. Quality similar to C300, but lens has wider angle as C300. Quality not better than C300.

Logitech Quickcam Pro for Notebooks

• Properties: 2.0 Mpixel, Karl-Zeiss-lens, autofocus or manual software driven, max. resolution: 1600x1200 @ 5 fps (YUV) or 960x720@15fps (MJPEG)
• Very small and inconspicious.
• 5fps too less as sidecam, suitable as frontcam

Common to all these cams

• Most Logitech webcams are supported by linux very well.

• TODO direct picture comparsion of cams
• Supported resolutions of logitech cama see <http://www.quickcamteam.net/devices/logitech_uvc_frame_format_list.pdf>

Ordanary still image camera

Fujifilm S9600 (640x480 @ 30fps) good enough as frontview on motorways: Street signs, lanes, limits cognizable. Not good enough for reading street names und house numbers. Main disadvantage: Abort after 2GB (->AVI, FAT), flash memory capacity limited and expencive. Still usable as an workaraound.

Keep in observation

HD-(still- or movie-)Cams, Canon cams: Hacked Firmware.

Views

Front, Back, Left, Right (F,B,L,R)

• Frontview: Most important. Steet-/navigation-/limit signs, POIs, lanes, side-attached house numbers.
• Rightview: Important to determine position of POIs mor accurate („which timestamp the GPS is in height of POI), house numbers, branches of small ways that can be overseen easily otherwise and so on
• Important enrichement.
• Leftview: Recommendable. Can be omitted if same trip is driven backward (not necesseraly possible!). Usage analog to right view.
• Backview: Not installed, but could be helpful to get more side-attached house numbers. Can be omitted if same way driven back..
• Also think of: Diagonal views. Pro: Needs less cameras. Contra: Objekt positionen can not be determined accurate.

Camera-Installation

• Hidden-secretly / demonstrative-provoking?
• Care of polarized, light killing glass in modern cars (best open windows)
• Care of mirroring objects and dust in glass
• TODO: Photos and comments of my personal installation

other Hardware (TODO Schema graph):

• car supply → inverter 12V - 230V → notebook power supply → notebook
• Notebook → Camera 1 (USB)
• Notebook → Camera 2 (USB)
• ...further Cameras…
• Notebook → GPS (USB)
• GPS → external Antenna
• OBD → serial
• USB → notebook (Inkremental sensors of front wheel) realizable?

TODO Hints for mounting cables

Software

• guvcview to record the video streams; runs in parallel multiple instances; setting save- and loadable as profiles; well communication with program author.
• TangoGPS to record gps-track; Trip function can be used „where have we already been today“, OpenStreetBUgs loadable as POIs
• Could be combined with more features in own software in the long term. Timestamps of video and GPS can easily kept synchronized due unique computer system time (no need for complicated manual sync).

Filecontainerformat/Compression

AVI: NoGo! Limited to 2GB (specification), longest 4GB (technical, 32Bit-Pointer). MKV: Pros: Timestamp per frame (nice for synchronization), metadata per frame (GPS-Position could be allocated to each frame!), open and documented format (fits mood of osm). Cons: Not supported by every software. Power-Fail produces file corrupt and enforces therefore a complicated restore procedure, because frameindex is not been written at end of file (← maybe program author could provide a solution)

MPEG2, Theora, H264 definitely to slow or to worse quality for realtime-HD-encoding on today's hardware uncompressed YUV, RGB needs too much disk space (think of USB/Harddisk transfer rates, too) MJPG best compromise; beeing delivered by many WebCams directly (no further CPU usage)

Ideal weather conditions

Sunshine (→ light!, GPS-receiption) (TODO: Test rainy/cloudy day, optimize guvcview-parameters)

Executing tours

• Alone: Possible. Keep notebook out of sight of driver because of safety, police.
• With codriver: Probably very advisable (techical supervising; route planning „where have we already been“ „where is the next bug“; temporally as „walker“ for footway and so on., technical equipment has to be supervised against theft; have more fun; change of roles possible)
• Problem bicycle: Energy supply?

Conclusion

Positive surprising results. MultiCam-Videomapping can be a good solution for fetching almost all details also as for bulk mapping far-away areas. Precondition is having fun evaluating video material in slow motion - will probably multiple times than the survey itself. Teamwork is recommandable - cams can be lend to other people while evaluating.