Autonomous Driving & ADAS

Structured Roads Are Not the Easy Case They Appear to Be

Autonomous driving systems trained on a single dataset — KITTI, nuScenes, or Waymo Open Dataset — generalize poorly across the US and EU operational domains. This is not purely a data volume problem; it is a distributional shift problem. US highways feature high-speed merges, wide lane markings, and aggressive lane-change behavior, while European roads add roundabouts, narrow medieval town centers, dense cyclist traffic, dynamic speed limits, and harmonized but country-specific signage. Add winter conditions — snow-covered lane markings in the US Midwest and Scandinavia, salted-road glare — and the perception and planning stack must treat the full US/EU scenario distribution as a first-class requirement.

Safety-critical disclaimer: ADAS and autonomous driving systems directly affect occupant and road-user safety. All production deployments must pass FMVSS / UNECE type approval, and any AI-based safety system requires formal verification, fault-mode analysis (FMEA/FTA), and conformance to ISO 26262 functional safety and ISO 21448 (SOTIF) standards, with UL 4600 used as the safety case framework for fully autonomous products. The techniques in this chapter are for engineering development; do not deploy perception or planning AI in public-road vehicles without completing the full homologation process.

Perception Pipeline Architecture

A production-grade perception stack takes raw sensor data and produces a structured world model: detected objects with class, position, velocity, and uncertainty; driveable surface estimates; and traffic sign/signal states.

Sensor Modalities

For US/EU mass-market programs, the camera + radar combination (no LiDAR) is the pragmatic entry point for Level 2+ ADAS — it matches the cost envelope of mainstream segments (Ford, Stellantis, VW C/D-segment vehicles). LiDAR is reserved for L3/L4 programs (Mercedes Drive Pilot, robotaxi development at Waymo and Cruise).

Object Detection: Camera Branch

The backbone of camera-based detection for automotive is BEV (Bird's Eye View) perception, where multiple camera views are transformed into a unified top-down representation. Key architectures:

For US/EU OEM programs, the practical constraint is the ADAS ECU compute budget — Mobileye EyeQ5 / NVIDIA Orin class processors. BEVDepth variants with INT8 quantization fit within the 20–40 TOPS envelope available.

# Inference pipeline structure for a typical camera-radar ADAS stack
# Open data/adas-perception-logs.json for sample US/EU traffic scenarios

import json

with open("data/adas-perception-logs.json") as f:
    scenario = json.load(f)["scenarios"][0]  # "Autobahn merge, motorcycle filtering"

# Each scenario has:
# - camera_frames: list of image paths
# - radar_tracks: [{id, range_m, azimuth_deg, velocity_mps, rcs_dbsm}]
# - ground_truth: [{object_class, position_xyz, velocity_xyz, bbox_3d}]
# - scenario_tags: ["motorcycle", "high_speed_merge", "highway", "peak_hour"]

print(f"Scenario: {scenario['description']}")
print(f"Objects: {len(scenario['ground_truth'])}")
print(f"Tags: {scenario['scenario_tags']}")

Sensor Fusion Architectures

Sensor fusion combines measurements from multiple sensors to produce estimates more accurate and robust than any single sensor alone. There are three architectural patterns:

Early Fusion (Raw Data Fusion)

Concatenate raw or near-raw sensor data before the detection network. Camera feature maps and LiDAR point clouds are projected into a common BEV representation and processed jointly. Best accuracy, highest compute, hardest to debug.

Mid Fusion (Feature Fusion)

Each sensor branch extracts features independently; features are fused at an intermediate layer. More modular — camera branch can be updated without retraining LiDAR branch. Current industry standard for camera+LiDAR systems.

Late Fusion (Track Fusion)

Each sensor branch produces tracked objects independently; an association algorithm merges the track lists. Most interpretable, easiest to validate for functional safety (each branch testable independently), but misses cross-modal correlations.

For ISO 26262 ASIL-B/D compliance, late fusion with redundant independent channels is preferred — each channel can be individually certified, and the fusion layer implements a voter/monitor architecture. SOTIF (ISO 21448) analysis additionally drives the handling of "known unsafe" and "unknown unsafe" perception scenarios that functional safety alone does not address.

Kalman Filter vs Neural Track Fusion

The Extended Kalman Filter (EKF) has been the workhorse of sensor fusion for 30 years. Neural approaches (Transformer-based tracking like MUTR3D, MOTR) show better performance on complex scenarios but are harder to bound formally:

For ADAS L2 production programs at US/EU OEMs, EKF with radar-primary tracking and camera-based object class enrichment is the validated path. Neural fusion is appropriate for L4 research programs.

Path Planning for US/EU Traffic

The standard Frenet-frame path planning (sampling-based or optimization-based lateral/longitudinal profiles) works well for structured highway scenarios. European urban scenarios require specific extensions:

Roundabouts and Unsignalized Negotiation

Roundabouts are pervasive in the EU (and increasingly in the US) and require yield-on-entry gap acceptance rather than signal obedience. A naive rule-based planner will either deadlock (perpetually yielding into continuous flow) or enter aggressively. Learned planners trained on real roundabout-entry data handle the implicit gap negotiation better:

Prompt: "I am designing a path planner for multi-lane roundabouts in the EU.
Common scenarios: cyclist in the roundabout ring on a shared lane, motorcycle
filtering on entry, pedestrian at the exit zebra crossing.

Design the state machine for the roundabout approach:
- Define the states, transitions, and timeout conditions
- Specify what each sensor input maps to state transitions
- Identify the minimum gap acceptance criteria for each opposing traffic type
- Flag which conditions require mandatory stop vs. creep-and-assess"

Snow-Covered Lane Marking Detection

Winter conditions in the US Midwest and Northern Europe obscure lane markings under snow and salt. Radar does not detect markings. Camera-based lane detection degrades sharply. Key ML task: robust drivable-corridor estimation that fuses prior map geometry, wheel-track inference from the lead vehicle, and road-edge detection when painted markings are absent.

FMVSS and UNECE Homologation

In the US, ADAS functions are governed by NHTSA under the Federal Motor Vehicle Safety Standards (FMVSS), while the EU and UNECE-aligned markets require UNECE type approval. The relevant regulations:

In the EU, the General Safety Regulation (GSR2) mandates AEB, intelligent speed assistance, lane-keeping, and driver drowsiness detection on new vehicle types from 2022 and all new registrations from 2024. UNECE R152 test scenarios use Euro NCAP-aligned protocols; US FMVSS 127 specifies its own pedestrian and lead-vehicle test matrix with nighttime conditions.

V2X: Vehicle-to-Everything Communication

V2X (Vehicle-to-Everything) is the communication infrastructure that allows vehicles to share position, speed, and intent data with each other (V2V), with infrastructure (V2I), and with pedestrians (V2P). The US/EU approach:

For ADAS engineers, V2X data augments onboard perception: an intersection controller can broadcast signal phase and timing (SPaT messages) 300m before the intersection, enabling predictive deceleration that pure camera systems cannot achieve.

Key Takeaways