Notes

Last updated Aug 10, 2022

Metadata

• URL:: https://www.coursera.org/learn/intro-self-driving-cars
• Tags:: #on/Coursera/Specialization, #on/Autonomous-Driving
• Links:: Self-Driving Cars Specialization
• Status:: #in-progress
• Instructor:: Steven Waslander, Jonathan Kelly
• Created:: 2021-08-05 09:34:56
• Modified:: 2022-05-13 16:46:43

• Driving Task
• Careful planning for Operational Design Domain (ODD)
• To classify driving system automation, we need to clearly define the requirements for driver attention, driver action, and driving tasks.
• Taxonomy of driving automation
• Requirements for perception
• Driving decisions and actions

• Sensors and computing hardware for AVs
• Software Architecture
• Let’s take a look at a representative modular software architecture for self driving cars, which will use the information provided by the hardware components and allow us to achieve our goal of autonomous driving.
  • 
  • The raw sensor measurements are passed into two sets of modules dedicated to understanding the environment around the car.
  • The Environment Perception module has two key responsibilities:
    • Identify the current location of the AV.
    • Classifying and locating important elements of the environment for the driving task.
  • The Environment Mapping module creates a set of maps which locate objects in the environment around the AV for a range of different uses.
  • The Motion Planning module makes all the decisions about what actions to take and where to drive based on all of the information provided by the perception and mapping modules.
    • The motion planning module’s main output is a safe, efficient and comfortable planned path that moves the vehicle towards its goal.
  • The Controller module takes the path and decides on the best steering angle, throttle position, brake pedal position, and gear settings to precisely follow the planned path.
  • The System Supervisor monitors all parts of the software stack, as well as the hardware output, to make sure that all systems are working as intended. It is also responsible for informing the safety driver of any problems found in the system.
• Environment Perception Module
  • 
• Environment Mapping Module
  • 
  • It creates different representations of current environment around the AV.
  • The Occupancy Grid Map is a map of all static objects in the environment surrounding the vehicle.
    • It represents the environment as a set of grid cells and associates a probability that each cell is occupied. This allows us to handle the uncertainty in the measurement data and improve the map over time.
  • The Localization Map is used by the Localization Module (in the Environment Perception module) in order to improve eco state estimation.
    • Sensor data is compared to this map while driving to determine the motion of the car relative to the localization map. This motion is then combined with other proprioceptor sensor information to accurately localize the eco vehicle.
  • The Detailed Road Map provides a map of road segments which represent the driving environment that the autonomous vehicle is currently driving through.
    • It captures signs and lane markings in a manner that can be used for motion planning.
    • This map is traditionally a combination of prerecorded data as well as incoming information from the current static environment gathered by the perception stack.
• The environment mapping and perception modules interact significantly to improve the performance of both modules.
• Motion Planning Module
  • 
  • As it is a challenging task, most self-driving cars today use a decomposition that divides the problem into several layers of abstraction.
  • The Mission Panner handles long term planning and defines the mission over the entire horizon of the driving task, from the current location, through the road network to its final destination.
  • The Behavior Planner solves short term planning problems. It is responsible for establishing a set of safe actions or maneuvers to be executed while travelling along the mission path.
  • The Local Planner performs immediate or reactive planning, and is responsible for defining a specific path and velocity profile to drive.
    • The local plan must be smooth, safe, and efficient given all the current constraints imposed by the environment and maneuver.
    • The output of the local planner is a planned trajectory which is a combined desired path and velocity profile for a short period of time into the future.
• Vehicle Controller
  • 
  • A typical vehicle controller takes the given trajectory and turns it into a set of precise actuation commands for the vehicle to apply.
  • It separates the control problem into a longitudinal control and a later control.
    • The lateral controller outputs the steering angle required to maintain the planned trajectory.
    • The longitudinal controller regulates the throttle, gears and braking system to achieve the correct velocity.
• System Supervisor Module
  • 
  • It continuously monitors all aspects of the autonomous car and gives the appropriate warning in the event of a subsystem failure.
  • There are two parts,
    • The hardware supervisor
      • The hardware supervisor continuously monitors all hardware components to check for any faults, such as a broken sensor, a missing measurement, or degraded information.
      • Another responsibility of the hardware supervisor is to continuously analyze the hardware outputs for any outputs which do not match the domain which the self-driving car was programmed to perform under.
        • For example, if one of the camera sensors is blocked by a paper bag or if snow is falling and corrupting the LIDAR point cloud data.
    • The software supervisor.
      • The software supervisor has the responsibility of validating this software stack to make sure that all elements are running as intended at the right frequencies and providing complete outputs.
      • The software supervisor also is responsible for analyzing inconsistencies between the outputs of all modules.
• Environment Representation
  • Localization Map
    • This map is created using a continuous set of LIDAR points or camera image features as the car moves through the environment.
    • It is then used in combination with GPS, IMU and Wheel Odometry by the localization module to accurately estimate the precise location of the vehicle at all times.
    • 
    • The difference between LIDAR maps is used to calculate the movement of the autonomous vehicle.
      • As new LIDAR camera data is received, it is compared to the localization map and a measurement of the ego vehicle’s position is created by aligning the new data with the existing map.
      • This measurement is then combined with other sensors to estimate ego motion and ultimately used to control the vehicle.
  • Occupancy Grid Map
    • The occupancy grid is a 2D or 3D discretized map of the static objects in the environments surrounding the ego vehicle.
    • The occupancy grid also uses a continuous set of LIDAR points to build a map of the environment which indicates the location of all static, or stationary, obstacles.
    • This map is used to plan safe, collision-free paths for the autonomous vehicle.
    • 
    • It represents the environment probabilistically, by tracking the likelihood that a grid cells occupy over time.
    • As the occupancy grid only represents the static objects from the environment, all dynamic objects must first be removed.
  • Detailed Road Map
    • It contains detailed positions for all regulatory elements (traffic lights, traffic signs), regulatory attributes (right turn markings, crosswalks) and lane markings.
    • This map is used to plan a path from the current position to the final destination.
    • 
    • The detailed road map can be created in one of three ways
      • Fully Online
        • It relies heavily on the static object proportion of the perception stack to accurately label and correctly localize all relevant static objects to create the map.
        • This is rarely used due to the complexity of creating such a map in real time.
      • Fully Offline
        • It is usually done by collecting data of a given road several times by using specialized vehicles with high accuracy sensors, and a mixture of automatic labelling and human annotation and correction.
        • It is unable to react/adapt to a changing environment.
      • Created Offline and Updated Online
        • This method of map creation takes advantage of both methods, creating a highly accurate roadmap which can be updated while driving.

• Basic Safety Terms
  • Harm: Physical harm to a living thing.
  • Risk: Probability that an event occurs, combined with the severity of the harm.
  • Safety: Absence of unreasonable risk of harm.
  • Hazard: Potential source of unreasonable risk of harm.
• [?] What are most common sources of major hazards?
  • Mechanical, e.g., incorrect assembly of a brake system causing a premature failure.
  • Electrical, e.g., faulty internal wiring leading to a loss of indicator lighting.
  • Hardware, i.e., failure of computing hardware chips.
  • Software, i.e., due to errors or bugs in the autonomy software.
  • Sensors, e.g., bad or noise sensor data or inaccurate perception.
  • Behavioral, i.e., due to incorrect planning or decision making.
  • Fallback, i.e., fallback fails without providing enough warning to the driver to resume responsibility.
  • Cyber, i.e., AV gets hacked by some malicious entity.
• [?] What is the NHTSA Safety Framework?
  • In the US, the National Highway Transportation Safety Administration or NHTSA, has defined a twelve-part safety framework to structure safety assessment for autonomous driving.
    • This framework was released as a suggested, not mandatory, framework to follow in 2017.
    • It consists of 12 areas:
      • System design approach to safety
        • Well-planned and controlled software development processes are essential, and the application of existing SAE and ISO standards from automotive, aerospace, and other relevant industries should be applied where relevant.
      • Autonomy Design
        • Operational Design Domain (ODD)
          • Encourages a well-defined Operational Design Domain (ODD) so that the designers are well aware of the flaws of this and limitations of the system.
        • Object and Event Detection and Response (OEDR)
          • Encourages a well-tested Object and Event Detection and Response (OEDR) system, which is critical to perception and crash avoidance.
        • Fallback
          • Encourages the car to have a reliable and convenient fallback mechanism by which the driver is alerted or the car is brought to safety autonomously.
          • It is essential to develop this mechanism keeping in mind that the driver may be inattentive.
        • Traffic Laws
          • The driving system should also be designed such that all the federal level, state level, and local laws for traffic are followed and obeyed within the ODD.
        • Cybersecurity
          • Encourages designers to think about cybersecurity threats, and how to protect the driving system from malicious agents.
        • HMI
          • There should be some thought put into the human machine interface, or HMI, so the car should be able to convey the status of the machine at any point in time to the passengers or the driver.
          • Important examples of status information that can be displayed, are
            • whether all sensors are operational,
            • what the current motion plans are,
            • which objects in the environment are affecting our driving behavior, and so on.
      • Testing and Crash Mitigation
        • Testing
          • Recommends a strong and extensive testing program before any service is launched for the public.
          • This testing can rely on three common pillars:
            • simulation
            • close track testing
            • public road driving
        • Crashworthiness
          • There should be careful consideration of methods to mitigate the extent of injury or damage that occurs during a crash event.
        • Post Crash
          • There should be support for post crash behavior.
          • The car must be rapidly returned to a safe state, for example, brought to a stop with fuel pumps securing the fuel, first responders alerted, and so on.
        • Data recording
          • There should be an automated data recording function or black box recorder.
          • It is very helpful to have this crash data to analyze and design systems that can avoid the specific kind of crash in the future, and to resolve questions about what went wrong, and who was at fault during the event.
        • Consumer Education
          • There should be well-defined consumer education and training.
          • This final step is essential to ensuring our natural overconfidence in automation, does not lead to unnecessary hazards being introduced by early adopters.
• [?] What are the various approaches that can be used to assess the safety of an autonomous driving system?
  • Analytical Safety Assessment
    • Ensuring the system works in theory and meets safety requirements found by hazard assessment.
    • This method can only provide guidance on the safety performance of the self-driving system, and the results need to be validated through experience.
  • Data-Driven Safety Assessment
    • Safety guarantee due to the fact that the system has performed autonomously without fail on the roads for a very large number of kms.