BMW's hi-tech vision for driving brings us back to nature

The "old guard" auto manufacturers are fighting back!

Tesla has something to worry about. The establishment has shaken off its complacency and it's getting ready to compete with the progress and innovations brought by Elon Musk's startup. The German manufacturer BMW has presented a new way of interacting with its car in the Catalan capital. The Munich-based company wants to revolutionize the way drivers interact with their cars.

Called BMW Natural Interaction, the new system combines voice control technology with extended gestural control and facial recognition to enable true multimodal operation for the first time.

"Customers should be able to communicate with their intelligent connected vehicle in a totally natural way. People shouldn’t have to think about which operating strategy to use to get what they want. They should always be able to decide freely – and the car should still understand them." explains Christoph Grote, Senior Vice President at BMW.

Voice, gestures and gaze

As in a people-to-people conversation, BMW's new feature allows the driver to use his voice, gestures and gaze simultaneously to interact with his vehicle. As the manufacturer explains, this operating mode can be chosen intuitively, depending on the situation and context, and voice commands, gestures and gaze direction can be reliably detected by the vehicle.

It is also possible to voluntarily choose the mode of operation, based on personal preferences, habits or current situation. This means that when the driver is engaged in a conversation, they can choose to control with gestures and eyes, while when their eyes are on the road, it is better to rely on speech and gestures. This way, for example, the car windows or the sunroof can be opened or closed or the ventilation adjusted. If the driver wants to know more about the vehicle's functions, they can also point to the buttons and ask what's happening. Goodbye to the famous user manual!

On the technical side, BMW uses optimized sensor technology and contextual analysis of gestures. Of course, artificial intelligence and machine learning are part of the solution to learn driver behavior and progress, and an algorithm combines and interprets the information so that the vehicle can react accordingly. This creates a multimodal interactive experience tailored to the driver's wishes.

Interaction both inside and outside

But the intelligence of the system does not stop in the passenger compartment of the car. Thanks to better recognition of actions and the car's high level of connectivity, it is also possible to interact with the world outside the vehicle, such as buildings or parking spaces.

For example, it will be possible to ask the system: what is this building? How long has this company been in existence? What is the name of this restaurant? Can I park here? How much does it cost?

If the theory seems attractive, the practice is just as attractive. The demonstration in mixed reality that I participated in Barcelona was convincing - BMW Natural Interaction proved to be attractive. Now it remains to be tested in a production vehicle while driving.

The autonomous car as an objective

In the long term, BMW will obviously target self-driving cars with its new system. This is a logical continuation of the fact that the vehicle is becoming more intelligent and the driver freer. "BMW Natural Interaction is also an important step for the future of autonomous vehicles, when the interior concepts will no longer be geared solely towards the driver’s position and occupants will have more freedom." explains Christoph Grote.

In the meantime, the functionalities of BMW Natural Interaction will be available in the BMW iNEXT from 2021 onward.

Would you like to use a similar system in your vehicle? Let us know in the comments.