Lane Detection on FPGAs

Learn how to perform hardware-accelerated vision processing for driver assistance and automated driving systems by implementing real-time lane detection.

Download models (login required) Request free workflow demo

Learn How to

Use Vision HDL Toolbox™ to model a lane detection algorithm in Simulink^®
Auto-generate VHDL code from your Simulink^® model and deploy to Speedgoat Simulink^®-Programmable FPGAs
Process high-resolution video streams with high sample frequency in real-time
Directly access video I/O such as HDMI with low latency

Key Benefits

Build vision-based Advanced Driver Assistance Systems (ADAS) and automated driving systems with hardware-proven subsystems
Eliminate time-consuming and error-prone steps with automated HDL code generation
Quickly perform design iterations and test your algorithms continuously
Bridge the gap between algorithm development and hardware deployment with MATLAB^®, Simulink^®and Speedgoat hardware solutions

Reference Example

Description

In assisted and automated driving, perception of the vehicle’s surroundings is one of the first steps towards automated vehicle control. A key goal for highway lane following is to keep the vehicle centered in its lane while driving. This requires detecting the lane boundaries relative to the vehicle, for which a windshield-mounted camera can be used.

However, processing high-resolution, high frame rate video data is computationally very intensive. Also, because the task is safety-critical, you want to minimize any delay in corrective actions at highway speeds. Therefore, such demanding tasks are best outsourced to specialized, independent target devices such as FPGAs.


Passenger Vehicles	Trucks	Off-Highway Vehicles	Racecars

But why choose FPGAs for vision processing?

The key element is their parallel architecture and inherent concurrency. FPGAs allow you to run and pipeline multiple vision processing jobs in a single clock, thus resulting in ultra-low input and output latency. Furthermore, FPGAs are not required to schedule multicore execution like CPUs and GPUs and offer better deterministic behavior while being much more power-efficient.

Finally, the architecture of FPGAs is also much closer to ASICs (Application-Specific Integrated Circuits), which is still the most common fabric for medium to high volume ADAS windshield camera production applications. You therefore benefit from a cost-efficient path to deployment.

But why choose FPGAs for vision processing?

The main challenge in targeting FPGAs is typically bringing your algorithm into low-level hardware description language (HDL). This is where the streamlined workflow for programming Speedgoat FPGAs comes in. Simulink^®-Programmable FPGAs, together with HDL Coder™ and Simulink^® Real-Time™, allow you to automatically generate efficient HDL code directly from your Simulink^® model and easily define your vision interfaces.

Furthermore, Vision HDL Toolbox™ facilitates collaboration between image processing and hardware design engineers, allowing them to easily transition from conventional frame-based exploratory algorithms to performant pixel-streaming architectures that run on FPGAs.

Vision HDL Toolbox™ provides numerous reference examples that illustrate real implementations, allowing you to get started swiftly and focus on creating your vision application.

The HDL Workflow Advisor guides you through the process of generating HDL code from your Simulink^® model step-by-step. The workflow advisor checks your model, notifies you of any incompatibilities, and proposes possible solutions.

For example, it helps you to find optimal settings for inputs, outputs, and clock rates. The auto-generated FPGA model is then deployed to the Speedgoat Simulink^®-Programmable FPGA in just a few clicks with Simulink^® Real-Time™.

The Speedgoat Lane Detection on FPGA reference example demonstrates the Simulink^®implementation of a real-time lane detection algorithm supporting HDL code generation. It is based on the MATLAB^®Lane Detection example and is adapted to operate on a 1080p 60fps video stream while using the Speedgoat Simulink^®-Programmable FPGA Module IO333-325k-SFP.

Inside the FPGA, the HDMI input image feed undergoes a set of processing steps such as birds-eye-view transformation, convolution with a vertically oriented Gaussian derivative filter, followed by gray-scale thresholding, which highlight the lane markings. Finally, the currently active lane is extracted while discarding outlier detections.

All these steps are executed concurrently as the frames are streamed in pixel by pixel, and only the ego lane detections are parsed to the software layer for final overlay (as illustrated in the video). Alternatively, you may want to feed the lane detector metadata to your lane-keeping controller and validate perception with controls.

For a more detailed introduction to the algorithm, please refer to the MATLAB^®Lane Detection example and the five-part video series Vision Processing for FPGA.

The Author

Timo Strässle
Application Engineer

Product Highlights

Speedgoat Products

MathWorks® Products

Vision HDL Toolbox™

Vision HDL Toolbox™ comes with several examples and reference examples that showcase how to apply the tools for image processing, video, and computer vision applications. For instance, you can master examples like noise removal and edge detection and apply these techniques to tasks such as lane detection, pothole detection, and image rectification.

Frequently, a vision processing pipeline has specific components that are particularly suited for FPGA deployment, which can lead to vast performance improvements. Use the well-known model-based design approach by Simulink^® to design vision algorithms that run on Speedgoat Simulink^®-Programmable FPGAs, and automatically generate efficient VHDL code using HDL Coder™.