SLAM 3D Image Processing – Electronic design services with Argon Design Ltd.

3D Image Processing

Skill sets used

Multimedia, Image and Video Processing, Portable products, Algorithms and mathematics

The Challenge

Much has been accomplished in the field of image processing and computer vision by analysing images as flat two dimensional data sets. To get to the next level though, images need to be treated as the projections of 3D scenes that they really are.

Sufficient computing power is now available to do this on desktop computers and even games consoles and there is currently keen academic research in this area. However, many of the algorithms being developed require huge amounts of computing power and have to run on clusters of GPUs.

Argon Design’s expertise is the use of 3D processing in embedded systems and mobile devices such as smart phones and tablets, where only moderate computing power is available. The key to this is using the right algorithms and detailed optimisation of code.

For embedded systems, choosing the right hardware arrangement can enable more ambitious processing which will give improved system performance and resilience - for instance a hardware front-end using FPGA logic, a low-power manycore processor or a mobile processor with a built-in GPU.

EKF SLAM System

As a demonstration of our capabilities, Argon Design has developed a system that can, in real-time, map a 3D scene as a camera is moved around it. The system accurately tracks the position and orientation (the "pose") of the camera with respect to the objects in the scene as well as finding the 3D position of key points on these objects in the image.

The system uses the Extended Kalman Filter Simultaneous Location and Mapping (EKF SLAM) technique.

A demonstration is shown in the following video:

Image data from the camera is supplemented by using the orientation data from the accelerometers, gyros and magnetometer as typically found on modern smart phones.

The first part of the video shows the process of identifying features to track and tracking them from frame to frame while the camera is moved around in both space and pointing direction. Red dots indicate features which are in good agreement with the predicted camera pose. Green dots are features found within their region of possible error. Blue dots are features found outside of their bounds and are not included in the camera pose update. The green ellipse around each feature indicates the search area over which the feature is searched for.

The second part of the video shows one example of how camera pose information can be used to add augmented reality objects to the scene. Note how as the camera is moved around, the Argon cube stays fixed in 3D space.

Applications

There are many applications of 3D image processing – some using single cameras as this example does and others using multiple cameras or structured illumination to provide more accurate 3D information.

Emerging applications for this technology are -

  • In an automotive application the locations of objects and hazards around a car could be used to assist manoeuvring
  • In face recognition, the recognition algorithm could be based on 3D models rather than 2D models allowing people to be recognised at any angle to the camera
  • In augmented reality, virtual objects can be added into a view of a 3D scene
  • In medical imaging, data can be analysed and visualised in new ways
  • In sport, actions can be viewed from an arbitrary viewpoint to fully analyse performance
  • In a robotics application, the system can make the robot aware of its surroundings to help it navigate and interact with its environment

The Future

We believe the use of 3D for image capture and processing will be one of the defining technological advances of the next decade enabling machines to do a lot more in the real world. In addition, we will see a move in multimedia from ‘pictures’ to 3D experiences using the techniques of virtual reality and augmented reality to get ever closer to a feeling of being there.

Related case studies

Contact us

Do you have a project that you would like to discuss with us? Or have a general enquiry? Please feel free to contact us

Contact us