Artificial intelligence has the potential to turn today’s simple drones into indispensable robot partners.
New technology from such companies as Santa Clara-based Nvidia is equipping drones with the capability to operate and perform complex tasks on their own. Industry insiders believe that this could pave the way for drones to be widely employed in large numbers in commercial applications, from delivering packages to inspecting infrastructure to serving as robot waiters.
Nvidia is a part of a wave of tech firms hoping to join the drone market. Companies ranging from technology giants like Intel and Apple to startups like Matternet are offering everything from unmanned aerial vehicles themselves, to enabling technologies, to complete drone-delivery services.
Smart chips for smart drones
The Jetson TK1 is a part of a line of platforms from Nvidia that are designed to bring artificial intelligence (AI) to devices like drones. Specifically, Nvidia is offering AI deep learning, which endows drones with the capability to recognize objects and navigate using visual inputs. Nvidia believes that its AI technology can eliminate many of the obstacles to the widespread commercial adoption of drones, also known as unmanned aerial vehicles (UAVs).
“Whether it’s surveying a field from above or inspecting a wind turbine, it’s much more efficient and safe to do it with a UAV. But for many applications for commercial UAVs today, they’re limited because of the requirements for GPS-based navigation," said Jesse Clayton, senior manager, product management, intelligent machines at Nvidia.
AI can address these concerns and limitations.
“The ability to navigate autonomously, to get in close to infrastructure and allow a drone to find the exact distance—that’s something deep learning can really help with,” Clayton noted. “The technology can also help drones to understand exactly what they are seeing.”
Onward with knowledge
Deep learning is an AI technique that gains knowledge through training a neural network, a computer system that's designed to process information like the human brain. As Nvidia describes it, a neural network can be taught to identify objects when it's shown many images of a single type of object, such as cars. Through this process, the network gains the ability to recognize cars and distinguish them from other objects.
The neural network can then recognize new objects it wasn’t trained on—such as other car models. This capability, called inference, allows devices like drones to identify objects out in the real world.
Illustrating how deep learning technology is used in drones during real-world applications, Clayton described how an AI-equipped drone could be used for inspecting a wind turbine.
With a conventional drone a technician travels to the turbine with a drone and flies the device around the wind turbine, hopefully without crashing into the wind turbine by accident. Next, an expert reviews the recorded data, and may ask the operator to take another flight to capture areas missed the first time. Finally, the maintenance is scheduled.
“It would be so much easier if the inspection could be scheduled, and the technician just goes out and presses a button and the drone would be intelligent enough to fly along the turbine by itself,” Clayton said. “The drone could fly close enough to get the footage that’s needed, while navigating accurately enough not to run into the turbine. It also can autonomously use artificial intelligence to understand what it’s looking at, so if it’s seeing a rust spot or damage it can generate the initial report right there on the drone."
Aerialtronics, a drone manufacturer, use Nvidia's Jetson for exactly that purpose, among other applications.
Warehouse inventory, smart video and more
Like Nvidia's Jetson, Intelligent Flying Machines Inc.’s flying robot is designed to take the place of humans in an but essential arduous task.
The IFM drone traverses warehouses to scan every item. Working autonomously, it can easily detect items that are inaccessible to people, such as packages located at the top of high shelving units. IFM said its system works at 400 times the speed of a conventional manual inventory process and can scan an entire warehouse in just 20 minutes, according to the IFM website.
"These IFMs can collect data fully autonomously and with centimeter accuracy – without the need for any external systems such as GPS," the Chicago-based company boasts.
Since GPS doesn’t work well in indoor environments, IFM developed a solution that combines a camera and sensors to fix the drone's position down to the centimeter.
The device uses onboard cameras, flight software and deep learning to keep track of people and its surroundings, according to a recent article in the MIT Technology Review.
Soaring around restrictions
Among the biggest obstacles preventing broad adoption of drones for commercial purposes are government regulations restricting their use. For example, a US Federal Aviation Administration rule requires operators to keep UAVs within their line of sight while airborne. This would preclude the deployment of proposed services that would use drones to deliver packages to remote places.
However, AI-equipped drones may help overcome government safety concerns.
“This technology can help drones perceive their environment and be successful in attaining goals like not crashing into things,” said Rick Rys, senior consultant at the market research firm ARC Advisory Group. “It also may help if a drone loses communications with a pilot, allowing it to complete its mission or simply return home.”
Nvidia predicts that efforts to commercialize self-driving cars will pave the legal path for drones.
“As far as regulatory adoption, I think you’re seeing a lot of industries go through this process—perhaps the automotive industry may be the first one to cross it,” Clayton said.
Keep your eyes on the sky.