Prove and test your model predictive lane following controller using real-time virtual vehicle simulation with raw synthetic camera data.

Watch video

See example model Request free workflow demo

Learn how to

Test controls and perception in closed-loop and real-time
Generate raw synthetic camera data for real-time perception emulation
Compute optimal control moves while accounting for system constraints (speed, acceleration, obstacles) using model predictive control
Frontload your controls and test with high-fidelity and real-time capable virtual vehicle simulation

Key Benefits

Integrate your algorithms for control or perception into the fully customizable reference example
Test prototype algorithms safely in a virtual environment while using real automotive interfaces for communication
Avoid expensive integration of hardware prototypes or real sensors by leveraging real-time sensor emulation
Simplify initial development of automated driving controllers with Model Predictive Control Toolbox™ and off-the-shelf blocks for adaptive cruise control, lane-keeping assistance, and path following

Get Started:

Overview

Virtual Vehicle Simulation

Controls Development and Interfaces

Product Highlights

Reference Example

Overview

Modern cars incorporate various control and assistance systems that influence the car's handling and driving experience. As important as each system's verification is, it is equally crucial to test them together and ensure they interplay seamlessly and stably.

A typical example is the development of more autonomous cars and the combination of perception and controls. Specifically, we will look at a model predictive lane following controller, which keeps the car centered in a highway lane, and a visual lane detection algorithm that detects highway lane locations using a simple windshield camera. On the one hand, controlling the car requires an accurate perception of the lane boundaries. On the other hand, the car's motion, which is stabilized by the control software, also influences the perception. The two systems can either support each other to reach stable steady-state performance or become unstable due to communication latency and other disturbances like outlier detections.


Passenger Vehicles	Trucks	Off-Highway Vehicles	Racecars

Virtual Vehicle Simulation

This reference example demonstrates how to test perception and controls in a closed-loop and run them on Speedgoat real-time target machines. Both systems are tested safely with virtual vehicle simulation. The whole scene is visualized in the Unreal Engine, generating synthetic camera images for the lane detection algorithm. This visualization allows us to verify the controller in a realistic and interactive environment, perform design iterations quickly, and log data for detailed analysis.

The reference example is based on the Highway Lane Following MATLAB^® example. While the original example is intended for desktop simulation, we have adapted it for real-time testing, therefore, splitting the application into the following four parts: The vehicle dynamics model, the lane detection algorithm, the lane following controller, and the visualization. The lane detection algorithm supports code generation and is deployed to a Speedgoat Performance real-time target machine alongside the vehicle dynamics model. We call this part the HIL-model (Hardware-in-the-Loop). The CPU-based vision algorithm achieves 10fps on a Speedgoat Performance real-time target machine, the same rate used in the original example, which is sufficient to validate the interaction of perception and controls. If the goal is to achieve higher frame rates at a later stage (e.g., 60fps), consider using an FPGA implementation.

Controls Development and Interfaces

The control algorithm is deployed to a second real-time target machine for Rapid Control Prototyping (RCP). The controller is based on a model predictive control scheme (MPC), allowing to enforce constraints such as the car's physical limitations and handling limits that ensure a comfortable driving experience. The control commands are then optimized over a finite, receding time horizon for which the car's behavior is predicted based on a dynamics model. Only the current timeslot is executed before optimizing again, enabling the controller to anticipate future events while at the same time remaining responsive to unforeseen disturbances. This control algorithm is easily implemented using a single block from Model Predictive Control Toolbox™.

Model Predictive Control Toolbox™ also supports integration with the FORCESPRO solver developed by Embotech AG. This plugin lets you generate resource-efficient, tailor-made, real-time solvers for complex MPC problems and run them reliably and deterministically on Speedgoat target hardware. To learn more about how to use this plugin with Simulink Real-Time™ and Speedgoat hardware, please refer to this lane following example by Embotech.

For visualization in Unreal Engine, dedicated Simulink® blocks are provided by the Automated Driving Toolbox™. Unreal Engine can only run on the host computer and requires a GPU for soft real-time performance. An additional Simulink^® block lets you easily generate a windshield camera image, routed to the HIL-system via an HDMI-to-USB frame grabber. The generated camera image can easily be replaced with real-world camera recordings. The interactive nature of closed-loop virtual vehicle simulation is ideal for testing ADAS functions, such as control handover between a human driver and the automated driving software.

The remaining data exchange goes through real-time UDP interfaces, with the option to use real automotive protocols instead, such as CAN or CAN-FD. This setup enables you to quickly iterate on different lane detection and motion control implementations and validate your modifications in real-time. Compared to prototype testing on the road, simulation allows for faster design iterations and eliminates the logistics effort and the risk of accidents and hardware damage. The real-time simulation further allows the interaction with real physical devices under test, such as an embedded controller or a vision processing unit. It lets you analyze the implementation's performance under realistic circumstances.

The Author

Timo Strässle
Application Engineer

Product Highlights

Speedgoat Products

MathWorks Products

Model Predictive Control Toolbox™

Model Predictive Control has proven to perform very well in practice. It incorporates physical constraints and uses a dynamics model to predict the system's behavior for a finite time horizon. Optimizing over a limited time window keeps the computational effort under control and allows real-time performance.

With Simulink^® and Model Predictive Control Toolbox™, you can design an MPC controller in a few clicks with MPC Designer, or drag-and-drop pre-built blocks for Lane Keeping or Cruise Control, using the default parameters or adjust them to your needs. The blockset supports code generation, allowing you to deploy and run your models in real-time on Speedgoat real-time target machines.

Product Focus: Model Predictive Control Toolbox™

FORCESPRO Solver

Users of FORCESPRO by Embotech and MathWorks' Model Predictive Control Toolbox™ can now leverage the computational performance of FORCESPRO from within Simulink^®, using the MATLAB^® plugin for FORCESPRO. The two products, combined with Speedgoat real-time target machines, provide a highly optimized environment to easily define challenging control problems and solve long-horizon MPC problems more efficiently, enabling you to optimally use the computational resources of your real-time system.