Potential for generative AI
Generative AI is a branch of artificial intelligence that can create novel and diverse data from existing data sources. It has the potential to enhance autonomous vehicle development and operations (AVOps) as the diagram shows.
Large language models and vision-enabled/multimodel foundation models can analyze the behavior of traffic participants and extract information from incoming frames/images (offline analysis). This data is stored in a scene library, which can be used for downstream processes such as training or validation datasets, or for creating new scenarios for simulations that are still not captured.
New enhancements to foundational models improve search recommendations with natural language queries that can measure the similarities between images and text, for example, to establish a scene library and improve coverage of the AD stack including long-tail scenarios.
You can augment the sensor data available from autonomous vehicles with the synthetic data generated by the AI models and enhance the perception, planning, and decision-making capabilities of the vehicles. This helps in improving the robustness and generalization of the systems.
Generative AI can help create realistic and diverse scenarios for testing and validation of autonomous driving systems. These scenarios greatly reduce the need for costly and time-consuming real-world data collection and annotation.
AI Copilots assist with various tasks such as rating, summarizing, elaborating, converting, and translating requirements and scenarios. Copilots can improve the quality, speed, and cost of software development by reducing errors and rework.
An AI pair programmer, like GitHub Copilot, can help write code not only faster than humans but also be more consistent with coding and safety standards for automotive requirements.
Security AI Copilots can help security teams protect their organizations from cyber threats at machine speed and scale. A Copilot works with the help of a large language model that can understand natural language queries and generate security-specific responses for an AVOps environment.
Based on these elements, a data-driven and generative AI-driven feedback loop can be established (as shown in the diagram) to make the development of AV (autonomous vehicles) models more efficient by increasing the coverage of potential scenarios and reducing manual efforts for identifying impactful datasets.