Skip to content

Occasionally interface event is lost #164

@yue-fred-gao

Description

@yue-fred-gao

This is seen once but it is worth to record the analysis.

  1. Below message is seen in the log: syncd vl_api_sw_interface_event_t_handler:632: vpp cannot get interface name for sw index 25, which means interface is not found in interface_name_by_sw_index.
  2. When an interface is up, vpp sends the interface-up event to the client (SONiC), which will be stored in a message queue. If it happens that refresh_interfaces_list is called before the event is picked up by vpp event polling thread (SwitchStateBase::vppProcessEvents is the handler), a race condition will happen causing this problem. Here are the details:
  • the interface_name_by_sw_index will be cleared first in api_sw_interface_dump
  • sending SW_INTERFACE_DUMP and PING API requests to vpp to get interface dump
  • calling W to process the responses from the message queue. If the interface up event is already received and currently stored in the queue, vl_api_sw_interface_event_t_handler callback will be invoked when processing the event. Because interface_name_by_sw_index is already cleared, we will end up the error message "vpp cannot get interface name".
  • Potential solution: call vpp_sync_for_events first in refresh_interfaces_list. This will process all the events in the queue. Ultimate solution is changing to some async model, which requires some investigation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions