Image Processing with the Jetson TX2s
While there are many factors which contribute to the success of an autonomous machine, they can be broadly separated into four areas: perception, mapping, path planning and actuation.
A key requirement for our driverless vehicle is a computer capable of providing sufficient processing power to achieve all these functions, whilst keeping energy consumption to a minimum.
As more processing power becomes available, the speed and efficiency of the autonomous system increases. Additionally, the lower the power consumption, the lighter the vehicle’s battery needs to be, and the longer it can drive without requiring a recharge. These two requirements often come at a trade-off, as a faster processor generally requires more power.
In order to meet these desired specifications, we decided upon a system with two main processing units. The first is the NVIDIA® Jetson™ TX2, whilst the other is a small form-factor Mini-ITX motherboard.
The first board, the NVIDIA Jetson TX2, will primarily be used for the cameras. The Jetson lends itself well to processing the data provided by the cameras due to the internal architecture of the processing unit.
The second board, what we’ve opted to simply call the ‘motherboard’, will be home to subsystems such as Path Planning, LiDAR, GPS, INS and Vehicle Actuation. These subsystems are all highly integrated and will require fast and efficient communication to reduce latency and increase the quality of the system.
In the computing world, there are two main types of processors: CPUs and GPUs. A CPU (Central Processing Unit) typically contains between 1-8 cores, where each core of the CPU is able to process data independently. On the other hand, a GPU (Graphics Processing Unit) can contain thousands of cores, each working independently on a large set of data. Combined with other optimizations within the processor, a conventional GPU can process trillions of numbers every second, compared to the few hundred billion of a conventional CPU.
At the heart of these optimizations is a parallel computing platform and application programming interface (API) called CUDA. Developed by NVIDIA in 2007, it allows us to write applications to be executed on the GPU. Contrary to common programming, which involves line-by-line execution of commands, CUDA allows you to define a set of commands which is then executed against a large set of data simultaneously. ‘Threads’ running these commands are assigned to individual cores within the processor, and allow us to leverage the full processing power of a GPU.
All these extra cores usually come at the expense of energy consumption. A high-end GPU can consume over 250 Watts of power, which is not ideal for a constrained application such as an autonomous vehicle. However, the Jetson TX2 does not suffer from this correlation. It contains 256 CUDA-enabled cores, and can perform over one trillion operations every second whilst only consuming 7.5 Watts of power, which is perfect for our application.
The primary purpose of the Jetson will be to process incoming stereo image pairs, and create an estimate of both the state of the vehicle, and the positions of cones on the track. Whilst the specifics behind this goal are still under investigation, it’s highly likely that a GPU optimized implementation of an algorithm called ORB-SLAM will be used for state estimation, in conjunction with a convolutional neural network, similar to YOLO v3 (You Only Look Once) for cone detection.
The output from the processing performed on the Jetson will then be piped to the ‘motherboard’. An algorithm will combine the estimations from the LiDAR, cameras and GPS to create a single highly accurate model of the world and the car, which can then be used by the path planning module to determine how to actuate the vehicle.
Our system is continuing to evolve as we progress further into our development and testing period. In the future, other subsystems may also be optimized for a GPU, and end up being implemented on a second Jetson TX2 to reduce power consumption and heat generation.
Our development would not have been possible without the support of NVIDIA and Xenon Systems. Their kind donation of two Jetson TX2’s has allowed us to leverage the full benefits of this low-power, high-performance GPU. If you are affiliated with an educational institution in Australia and have a valid accredited university email address, you would be eligible for a significant discount on the Jetson TX2 Developer Kit . For more information, visit www.xenon.com.au.
- Paras Bhutiani, James Wyatt