ELAM

ELAM Packet capture:

Step 1: Kindly check the hardware module to see what inputs must be there in command i.e. tah/roc/app

/// N9K-C*-EX- Leaf ----------------------- tah

N9K-C*FX/FXP/FX2 -Leaf ------------ roc

N9K-C*-GX – Leaf ----------------------app//

Recommended Best Practices:

• To capture a packet with VLAN encapsulation on a downlink port, use 'in-select 6'.

• To capture a packet with VXLAN encapsulation (whether from a spine or a vleaf with VXLAN encapsulation), use 'in-select 14'.

L3 traffic

ELAM JI

=========================

Leaf1

=========================

vsh_lc

debug platform internal tah elam asic 0

    trigger reset 

    trigger init in-select 6 out-select 1

      set outer ipv4 src_ip 192.168.11.11 dst_ip 192.168.12.12

      start

      status

      ereport

Python available. Continue ELAM decode with LC Pkg

ELAM REPORT

======================================================================================================================================================

Trigger/Basic Information

======================================================================================================================================================

ELAM Report File : /tmp/logs/elam_2024-06-29-02m-18h-46s.txt

In-Select Trigger : Outerl2-outerl3-outerl4( 6 )

Out-Select Trigger : Pktrw-sideband-drpvec( 1 )

ELAM Captured Device : LEAF

Packet Direction : ingress

Triggered ASIC type : Homewood

Triggered ASIC instance : 0

Triggered Slice : 0

Incoming Interface : 0x58( 0x58 )

( Slice Source ID(Ss) in "show plat int hal l2 port gpd" )

======================================================================================================================================================

Captured Packet

======================================================================================================================================================

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer Packet Attributes

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer Packet Attributes : l2uc ipv4 ip ipuc ipv4uc

Opcode : OPCODE_UC

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer L2 Header

------------------------------------------------------------------------------------------------------------------------------------------------------

Destination MAC : 0022.BDF8.19FF

Source MAC : ACF2.C5F8.2F81

802.1Q tag is valid : yes( 0x1 )

CoS : 0( 0x0 )

Access Encap VLAN : 3311( 0xCEF )

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer L3 Header

------------------------------------------------------------------------------------------------------------------------------------------------------

L3 Type : IPv4

IP Version : 4

DSCP : 0

IP Packet Length : 84 ( = IP header(28 bytes) + IP payload )

Don't Fragment Bit : not set

TTL : 255

IP Protocol Number : ICMP

IP CheckSum : 13018( 0x32DA )

Destination IP : 192.168.12.12

Source IP : 192.168.11.11

module-1(DBG-elam-insel6)# show plat int hal l2 port gpd

For Incoming interface

Incoming Interface : 0x58( 0x58 ) <-- Ss (Slice source)

For Outgoing Interface

module-1(DBG-elam-insel6)# report | grep ovec

  hom_elam_out_sidebnd_no_spare_vec.ovector_idx: 0x40

leaf1# show lldp neighbors int eth1/52

Device ID Local Intf Hold-time Capability Port ID

Spine1 Eth1/52 120 BR Eth1/29

========================

Spine1

========================

 vsh

attach module 1

debug platform internal roc elam asic 0

    trigger reset 

    trigger init in-select 14 out-select 1

      set inner ipv4 src_ip 192.168.11.11 dst_ip 192.168.12.12

      start

     module-1(DBG-elam-insel14)# status

ELAM STATUS

===========

Asic 0 Slice 0 Status Triggered

Asic 0 Slice 1 Status Armed

Asic 0 Slice 2 Status Armed

Asic 0 Slice 3 Status Armed

module-1(DBG-elam-insel14)# ereport

Python available. Continue ELAM decode with LC Pkg

ELAM REPORT

======================================================================================================================================================

Trigger/Basic Information

======================================================================================================================================================

ELAM Report File : /tmp/logs/elam_2024-06-29-18m-16h-32s.txt

In-Select Trigger : Outer(l2(vntag)|l3|l4)-inner(l2|l3|l4)-ieth( 14 )

Out-Select Trigger : Pktrw-sideband-drpvec( 1 )

ELAM Captured Device : SPINE_SC

Packet Direction : egress

Triggered ASIC type : Bigsky

Triggered ASIC instance : 0

Triggered Slice : 0

Incoming Interface : 0x58( 0x58 )

( Slice Source ID(Ss) in "show plat int hal l2 port gpd" )

Packet from vPC peer LEAF : yes

Packet from tunnel (remote leaf/avs) : yes

======================================================================================================================================================

Captured Packet

======================================================================================================================================================

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer Packet Attributes

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer Packet Attributes : l2uc ipv4 ip ipuc ipv4uc udp ivxlan

Opcode : OPCODE_UC

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer iEth Header

------------------------------------------------------------------------------------------------------------------------------------------------------

iEth SUP code : NONE

Packet from CPU : no

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer L2 Header

------------------------------------------------------------------------------------------------------------------------------------------------------

Destination MAC : 000D.0D0D.0D0D

Source MAC : 000C.0C0C.0C0C

802.1Q tag is valid : yes

CoS : 0

Access Encap VLAN : 2

VN-Tag is valid : no

<-- ( FC tells LC via Vn-Tag about which port the packet needs to go to )

Src VIF (in from leaf/IPN) : 0

<-- ( VIF(dec) in ELTMC or LID(hex) in "show plat int hal l2 port pi" )

Dst VIF (out to leaf/IPN) : 0

<-- ( OIFL in "show forwarding multicast outgoing-interface-list 0" )

------------------------------------------------------------------------------------------------------------------------------------------------------

Inner L2 Header

------------------------------------------------------------------------------------------------------------------------------------------------------

Inner Destination MAC : 000C.0C0C.0C0C

Source MAC : 000C.0C0C.0C0C

802.1Q tag is valid : no

CoS : 0

Access Encap VLAN : 0

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer L3 Header

------------------------------------------------------------------------------------------------------------------------------------------------------

L3 Type : IPv4

DSCP : 0

Don't Fragment Bit : 0x0

TTL : 32

IP Protocol Number : UDP

Destination IP : 10.0.248.0

Source IP : 10.0.32.69

------------------------------------------------------------------------------------------------------------------------------------------------------

Inner L3 Header

------------------------------------------------------------------------------------------------------------------------------------------------------

L3 Type : IPv4

DSCP : 0

Don't Fragment Bit : 0x0

TTL : 254

IP Protocol Number : ICMP

Destination IP : 192.168.12.12

Source IP : 192.168.11.11

module-1(DBG-elam-insel14)# show platform internal hal l2 port gpd

Incoming interface

Incoming Interface : 0x58( 0x58 )

Outgoing interface

module-1(DBG-elam-insel14)# report | grep ovec

bky_elam_out_sidebnd_no_spare_vec.ovector_idx: 0x148

module-1(DBG-elam-insel14)# show platform internal hal l2 port gpd | grep 148

1a000000 Eth1/1 0 2 3 0 2a 2 9 48 148 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 D-24d - 100 0 1 1 2 0 0

spine1# show lldp neighbors interface ethernet 1/1

Capability codes:

Device ID Local Intf Hold-time Capability Port ID

leaf3 Eth1/1 120 BR Eth1/49

========================

Leaf3

========================

debug platform internal tah elam asic 0

    trigger reset 

    trigger init in-select 14 out-select 1

      set inner ipv4 src_ip 192.168.11.11 dst_ip 192.168.12.12

      start

     module-1(DBG-elam-insel14)# status

ELAM STATUS

===========

Asic 0 Slice 0 Status Armed

Asic 0 Slice 1 Status Triggered

module-1(DBG-elam-insel14)# ereport

Python available. Continue ELAM decode with LC Pkg

ELAM REPORT

======================================================================================================================================================

Trigger/Basic Information

======================================================================================================================================================

ELAM Report File : /tmp/logs/elam_2024-06-29-24m-16h-34s.txt

In-Select Trigger : Outer(l2(vntag)|l3|l4)-inner(l2|l3|l4)-ieth( 14 )

Out-Select Trigger : Pktrw-sideband-drpvec( 1 )

ELAM Captured Device : LEAF

Packet Direction : egress

Triggered ASIC type : Sugarbowl

Triggered ASIC instance : 0

Triggered Slice : 1

Incoming Interface : 0x38( 0x38 )

( Slice Source ID(Ss) in "show plat int hal l2 port gpd" )

Packet from vPC peer LEAF : yes

Packet from tunnel (remote leaf/avs) : yes

======================================================================================================================================================

Captured Packet

======================================================================================================================================================

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer Packet Attributes

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer Packet Attributes : l2uc ipv4 ip ipuc ipv4uc udp ivxlan

Opcode : OPCODE_UC

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer iEth Header

------------------------------------------------------------------------------------------------------------------------------------------------------

iEth SUP code : NONE

Packet from CPU : no

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer L2 Header

------------------------------------------------------------------------------------------------------------------------------------------------------

Destination MAC : 000C.0C0C.0C0C

Source MAC : 0000.0000.0000

802.1Q tag is valid : yes

CoS : 0

Access Encap VLAN : 2

VN-Tag is valid : no

<-- ( FC tells LC via Vn-Tag about which port the packet needs to go to )

Src VIF (in from leaf/IPN) : 0

<-- ( VIF(dec) in ELTMC or LID(hex) in "show plat int hal l2 port pi" )

Dst VIF (out to leaf/IPN) : 0

<-- ( VIF(dec) in ELTMC or LID(hex) in "show plat int hal l2 port pi" )

-------------------------------------------------------------------------------------------------------------------------------

Inner L2 Header

-------------------------------------------------------------------------------------------------------------------------------

Inner Destination MAC : 000C.0C0C.0C0C

Source MAC : 000C.0C0C.0C0C

802.1Q tag is valid : no

CoS : 0

Access Encap VLAN : 0

------------------------------------------------------------------------------------------------------------------------------------------------------

Outer L3 Header

------------------------------------------------------------------------------------------------------------------------------------------------------

L3 Type : IPv4

DSCP : 0

Don't Fragment Bit : 0x0

TTL : 32

IP Protocol Number : UDP

Destination IP : 10.0.32.67

Source IP : 10.0.32.69

------------------------------------------------------------------------------------------------------------------------------------------------------

Inner L3 Header

------------------------------------------------------------------------------------------------------------------------------------------------------

L3 Type : IPv4

DSCP : 0

Don't Fragment Bit : 0x0

TTL : 254

IP Protocol Number : ICMP

Destination IP : 192.168.12.12

Source IP : 192.168.11.11

module-1(DBG-elam-insel14)# show platform internal hal l2 port gpd

Incoming Packet

Incoming Interface : 0x38( 0x38 )

Outgoing Interface

module-1(DBG-elam-insel14)# report | grep ovec

sug_elam_out_sidebnd_no_spare_vec.ovector_idx: 0x20

 ======================

Traffic from leaf3 to leaf1

=======================

debug platform internal tah elam asic 0

    trigger reset 

    trigger init in-select 6 out-select 1

      set outer ipv4 src_ip 192.168.12.12 dst_ip 192.168.11.11

      start

      status

report detail

      ereport

================================================

L2 traffic

================================================

vsh_lc

deb pla in <tah/roc/app> elam asic 0

trigger init in-select 6  out-select 1

set out l2 src-mac <Mac> dst-mac <mac address>   

start

status

report detail

ereport

===============================================

To capture and verify all three steps of the TCP handshake (SYN, SYN-ACK, and ACK) using ELAM on a Cisco ACI switch, you need to modify your ELAM trigger to capture each stage of the handshake.

Understanding TCP Flags for Handshake

1. SYN (Step 1): flags 0x02

2. SYN-ACK (Step 2): flags 0x12

3. ACK (Step 3): flags 0x10

   
   

   Steps to Capture the Full TCP Handshake Using ELAM

   Since vsh_lc commands allow debugging at the hardware level, you need to run three separate captures for each handshake step.

   
   

   1) Capture the SYN (First Step)

   
   

   vsh_lc

   debug platform internal roc elam asic 0

   trigger reset

   trigger init in-select 6 out-select 1

   set outer ipv4 src 10.230.126.2 dst 10.230.8.79

   set outer l4 l4-type 0 flags 0x02 # SYN

   start

   status

   report detail

   ereport

   
   

   • This captures the first SYN sent from the client (10.230.126.2) to the server (10.230.8.79).

   
   

   2) Capture the SYN-ACK (Second Step)

   
   

   vsh_lc

   debug platform internal roc elam asic 0

   trigger reset

   trigger init in-select 6 out-select 1

   set outer ipv4 src 10.230.8.79 dst 10.230.126.2

   set outer l4 l4-type 0 flags 0x12 # SYN-ACK

   start

   status

   report detail

   ereport

   
   
   
   

   • This captures the server's response (SYN-ACK).

   
   

   3) Capture the Final ACK (Third Step)

   
   

   vsh_lc

   debug platform internal roc elam asic 0

   trigger reset

   trigger init in-select 6 out-select 1

   set outer ipv4 src 10.230.126.2 dst 10.230.8.79

   set outer l4 l4-type 0 flags 0x10 # ACK

   start

   status

   report detail

   ereport

   
   

   • This captures the final ACK, completing the handshake.

   
   

   Also , you can use Tcpdump for TCP Handshake

   
   

   Leaf # tcpdump -i any host 10.230.126.2 and host 10.230.8.79 and tcp

   
   

   Summary

   • Run three ELAM triggers, each targeting flags 0x02, 0x12, and 0x10 respectively.

   • Verify handshake completion using the status command.

   • If needed, use SPAN/tcpdump to cross-check the handshake at a higher level.

===============================================

The 'ereport' command can be used to display ELAM results in a clear and easy-to-understand format.

ELAM reports are saved in the /var/log/dme/log/ directory on the switch. Each ELAM capture generates two files:

• elam_<timestamp>.txt

• pretty_elam_<timestamp>.txt

ftriage

Leaf # ftriage bridge -ii LEAF:104 -dmac 02:02:02:02:02:02

Leaf # ftriage route -ii LEAF:203,204 -sip 10.100.13.100 -dip 10.88.156.30

This command is using ftriage in its “route” mode. Here’s a breakdown of the components:

• ftriage routeThis tells the tool to perform a routing query.

• -ii LEAF:203,204The -ii flag specifies the interface indices. In this case, it targets the interfaces labeled “LEAF:203” and “LEAF:204.” These identifiers likely refer to specific network segments or physical interfaces that are part of the analysis.

• -sip 10.100.13.100The -sip flag indicates the source IP address for the routing query. This is the IP from which the traffic originates.

• -dip 10.88.156.30The -dip flag specifies the destination IP address. This is where the traffic is intended to go.

  
  

In summary, this command tells ftriage to check the routing path between the source IP (10.100.13.100) and the destination IP (10.88.156.30) while specifically considering the interfaces LEAF:203 and LEAF:204.

The output will likely show how packets are routed between these points across the given interfaces, which can be useful for troubleshooting connectivity or configuration issues in your network.

Additional Info

debug plat internal roc elam as 0

trigger reset

trigger init in-select 6 out-select 1

set outer ipv4 src_ip 0.0.0.0

set outer l2 src_mac 0050.5682.25D9

set outer l4 src-port 68

start

status

report detail

ereport

---------------------------------------------------------------------------------

ELAM JI

two files in /var/sysmgr/tmp_logs/

pretty_elam_...txt: The user-friendly, structured report for analysis.

elam_...txt: The raw, detailed report.

Traffic Scenario	in-select Value	Applied At
Endpoint → Leaf (L2 / L3 / VM / ICMP)	6	Leaf
VXLAN Overlay Traffic	14	Spine / Egress Leaf
ARP / ND Traffic	6	Leaf
L3Out Ingress Traffic	6	Border Leaf

When packets drop in ACI, stop guessing policy and ask the ASIC what it did.

ACI has:

Logical model (Tenant / EPG / Contract)

Control plane (COOP, routing protocols)

Data plane (ASIC forwarding) ← ELAM lives here

ELAM + ereport + HAL = absolute truth

Everything else (GUI, faults, health score) is secondary evidence.

ELAM → ereport → HAL

ELAM

Captures one real packet

Shows what the ASIC saw

Raw, ugly, but 100% accurate

ereport

Converts ASIC registers → human logic

Shows:

Forwarding lookup

Contract decision

Rewrite / drop vector

This is where you spend 80% of your time

HAL

Answers the question:

“What exactly is index 0x6011?”

HAL maps:

VRF IDs

Route indexes

BD IDs

ELAM says what happened

HAL explains why it was programmed that way

Steps for using ELAM

Step 1: Capture the right packet

in-select 6 → endpoint → leaf (front-panel)

in-select 14 → fabric traffic (VXLAN)

Also:

Always use out-select 1

→ without it, you lose drop vector & rewrite info (fatal mistake)

Step 2: Validate ingress immediately

In ereport:

Trigger / Basic Information

Incoming Interface (Ss)

Then HAL:

show platform internal hal l2 port gpd

Step 3: Confirm packet identity

Before troubleshooting:

Check MACs

Check IPs

Check VLAN

Check protocol/ports

Step 4 : Troubleshooting: FPB → FPC → RW

FPB – Forwarding Path Block (Routing & Bridging)

show platform internal hal l3 vrf pi

If wrong VRF → classification problem (EPG/BD/L3Out issue)

Was there a route?

Dst IP is Hit: yes/no

Dst IP Hit Index

show platform internal hal l3 routes

Hit = no → No route (UC_PC_CFG_TABLE_DROP)

Hit = yes but points to drop adj → Misprogrammed route

This is where “no route” vs “bad route” is differentiated.

FPC – Contract / Policy Block

Key fields:

sclass

dclass

Contract Result

ACLQOS index

If ereport shows:

SECURITY_GROUP_DENY

Then:

Convert ACLQOS index hex → decimal

show system internal aclqos zoning-rules | grep -B 9 "Idx: <decimal>"

Now you know:

Exact contract

Exact filter

Exact direction

This is bullet-proof contract debugging

RW / Sideband – Final Verdict

What did the ASIC finally do with the packet?

Key fields:

Lookup Drop

RW drop reason

ovector

RwEncapIdx

If:

Drop reason present → this node dropped it

No drop + ovector valid → packet forwarded → problem is downstream

“The leaf forwarded traffic correctly.”

How to Think About the Common Drop Scenarios

❌ SECURITY_GROUP_DENY

Meaning: Contract blocked it

Confirm:

Drop vector = SECURITY_GROUP_DENY

FPC shows deny

ACLQOS index → zoning rules

Policy issue, not routing, not fabric

❌ UC_PC_CFG_TABLE_DROP

Meaning: No usable route

Confirm:

FPB: Dst IP Hit = no

or

Route points to drop adjacency

L3Out / routing / subnet scope issue

❌ VLAN_XLATE_MISS

Meaning: VLAN not programmed on that port

Confirm:

Captured Packet shows VLAN X

Drop vector = VLAN_XLATE_MISS

👉 Static path / VMM / VLAN pool mismatch

❌ SMAC_MISS

Meaning: Source MAC not learned

Confirm:

Drop vector = SMAC_MISS

Check port security / MAC limit

---------------------------------------------------------------------------------------

Real ELAM Case Studies

CASE 1: Contract Drop (SECURITY_GROUP_DENY)

Customer Symptom

“App server (EPG-APP) cannot reach DB server (EPG-DB) on TCP/1521.”

Ping may work, TCP fails.

ELAM / ereport Findings

Captured Packet

SIP: 10.1.1.10

DIP: 10.2.2.20

Protocol: TCP

Dst Port: 1521

Forwarding Lookup (FPB)

VRF: Tenant-A:VRF1

Route Hit: YES

Dst EPG resolved correctly

Contract Lookup (FPC)

sclass: 16385 (EPG-APP)

dclass: 16390 (EPG-DB)

Contract Result: DROP

SECURITY_GROUP_DENY: YES

Aclqos Index: 0x13FB8

Rewrite / Drop

Lookup Drop Reason: SECURITY_GROUP_DENY

Interpretation

Routing is correct

Classification is correct

Drop happened only at policy stage

This is not a fabric or routing issue

Convert ACL index:

0x13FB8 = 81848 (decimal)

HAL / zoning-rule lookup:

show system internal aclqos zoning-rules | grep -B 9 "Idx: 81848"

→ Missing TCP/1521 filter in contract.

Root Cause

Contract between EPG-APP and EPG-DB does not permit TCP/1521.

Fix

Add correct filter → associate to contract → redeploy.

Explanation

“The switch received the packet, found the destination, but intentionally dropped it because the configured security policy does not permit this traffic.”

CASE 2: No Route (UC_PC_CFG_TABLE_DROP)

“Endpoint cannot reach external subnet via L3Out.”

ELAM / ereport Findings

Dst IP Lookup VRF: Tenant-B:VRF2

Dst IP is Hit: NO

Lookup Drop Reason: UC_PC_CFG_TABLE_DROP

Interpretation

Packet classified into correct VRF

No route exists in hardware

This is not adjacency, not policy

Check HAL:

show platform internal hal l3 routes vrf <VRF_ID>

→ Destination prefix missing.

Root Cause

External subnet not imported / advertised into VRF (missing subnet or BGP/OSPF issue).

Fix

Correct L3Out subnet scope or routing protocol configuration.

“The leaf switch dropped the packet because it has no route to reach the destination network.”

CASE 3: No Adjacency (ARP / ND Failure)

Customer Symptom

“Inter-EPG traffic works sometimes, fails intermittently.”

ELAM / ereport Findings

Forwarding Lookup

Route Hit: YES

Next-Hop L2 Ptr Valid: NO

MAC Lookup

Dst MAC Hit: NO

Rewrite / Drop

Lookup Drop Reason: NO_ADJ

Interpretation

Routing is correct

But leaf does not have MAC/ARP/ND for next-hop

Often caused by:

ND suppression issues

Endpoint flaps

COOP stale entries

Root Cause

Leaf hardware has no adjacency for the next-hop.

Fix

Verify ARP/ND

Check endpoint stability

Clear adjacency if needed

Customer Explanation

“The switch knows where the destination network is, but does not know how to reach the next device at Layer 2.”

CASE 4: VLAN_XLATE_MISS

Customer Symptom

“Endpoint traffic dropped immediately on ingress.”

ELAM / ereport Findings

Captured Packet

Ingress VLAN: 345

Rewrite / Drop

Lookup Drop Reason: VLAN_XLATE_MISS

Interpretation

VLAN not programmed on that leaf port

Static path / VMM binding mismatch

Root Cause

VLAN not allowed or not deployed on ingress interface.

Fix

Correct static binding or VLAN pool association.

Customer Explanation

“The switch dropped the packet because the VLAN is not configured on that port.”

Drop Vector	Description
SECURITY_GROUP_DENY	Traffic dropped due to contract deny
UC_PC_CFG_TABLE_DROP	No matching route found
NO_ADJ	ARP / ND resolution failure
VLAN_XLATE_MISS	VLAN not programmed on the leaf
SMAC_MISS	Port security or MAC address limit hit

-----------------------------------------------------------------------------------------------------

ACI ELAM / ereport Practice Worksheet

Question: “What is the root cause?”

Scenario 1

================= Captured Packet =================

Src IP: 10.1.1.10

Dst IP: 10.2.2.20

Protocol: TCP

Dst Port: 1521

================= Forwarding Lookup (FPB) =================

Dst IP Lookup VRF: 0x20003

Dst IP Hit: YES

Dst IP Hit Index: 0x6011

================= Contract Lookup (FPC) =================

Sclass: 16385

Dclass: 16390

Contract Result: DROP

SECURITY_GROUP_DENY: YES

Aclqos Stats Index: 0x13FB8

================= Rewrite / Drop =================

Lookup Drop Reason: SECURITY_GROUP_DENY

Analysis

Contract Drop

Drop Vector: SECURITY_GROUP_DENY

Root Cause: Missing or incorrect contract/filter between source and destination EPGs

Fix / Verify:

Convert ACLQOS index hex → decimal

show system internal aclqos zoning-rules

Verify contract direction and filter ports

Routing works, policy blocked it.

Scenario 2

================= Captured Packet =================

Src IP: 10.10.10.10

Dst IP: 8.8.8.8

Protocol: ICMP

================= Forwarding Lookup (FPB) =================

Dst IP Lookup VRF: 0x20004

Dst IP Hit: NO

================= Rewrite / Drop =================

Lookup Drop Reason: UC_PC_CFG_TABLE_DROP

Analysis

No Route

Drop Vector: UC_PC_CFG_TABLE_DROP

Root Cause: No route to destination in VRF

Fix / Verify:

Check L3Out subnets

Verify routing protocol (BGP/OSPF)

show platform internal hal l3 routes

Packet never found a route.

Scenario 3

================= Captured Packet =================

Src IP: 10.1.1.10

Dst IP: 10.3.3.30

================= Forwarding Lookup (FPB) =================

Dst IP Lookup VRF: 0x20003

Dst IP Hit: YES

Dst IP Hit Index: 0x6055

NextHop L2 Ptr Valid: NO

================= MAC Lookup =================

Dst MAC Hit: NO

================= Rewrite / Drop =================

Lookup Drop Reason: NO_ADJ

Analysis

No Adjacency

Drop Vector: NO_ADJ

Root Cause: ARP/ND unresolved for next-hop

Fix / Verify:

show ip arp vrf <vrf>

Check endpoint / firewall reachability

Look for endpoint flaps or ND suppression issues

Route exists, but next-hop MAC is missing.

Scenario 4

================= Captured Packet =================

Src MAC: 00:aa:bb:cc:dd:ee

Ingress VLAN: 345

================= Forwarding Lookup =================

VLAN Translation: MISS

================= Rewrite / Drop =================

Lookup Drop Reason: VLAN_XLATE_MISS

Analysis

VLAN Not Programmed

Drop Vector: VLAN_XLATE_MISS

Root Cause: VLAN not bound to EPG on ingress port

Fix / Verify:

Static path binding

VLAN pool association

Port and domain mapping

Dropped immediately at ingress.

Scenario 5

================= Captured Packet =================

Src MAC: 00:aa:bb:cc:dd:ff

VLAN: 100

================= MAC Lookup =================

Src MAC Hit: NO

Src MAC Learn Allowed: NO

================= Rewrite / Drop =================

Lookup Drop Reason: SMAC_MISS

Analysis

SMAC Miss

Drop Vector: SMAC_MISS

Root Cause: Port security / MAC limit exceeded

Fix / Verify:

Check MAC limit on EPG/static path

Disable or increase MAC limit

Verify number of learned MACs

ASIC refused to learn source MAC.

Self-Check Rule (Memorize This)

Route hit + policy drop → CONTRACT

Route miss → NO ROUTE

Route hit + no MAC → NO ADJ

Dropped at ingress → VLAN

SMAC not learned → PORT SECURITY

Mixed / Trick ELAM Practice Worksheet

Scenario 1 – Policy vs Routing Trap

================= Captured Packet =================

Src IP: 10.1.1.10

Dst IP: 10.2.2.20

Protocol: TCP

Dst Port: 1521

================= Forwarding Lookup (FPB) =================

Dst IP Lookup VRF: 0x20003

Dst IP Hit: YES

Dst IP Hit Index: 0x6011

================= Contract Lookup (FPC) =================

Sclass: 16385

Dclass: 16390

Contract Result: DROP

SECURITY_GROUP_DENY: YES

================= Rewrite / Drop =================

Lookup Drop Reason: SECURITY_GROUP_DENY

Extra Info from Customer

“We also noticed there is no route in the routing table sometimes.”

Contract Wins Over Routing

Primary Drop: SECURITY_GROUP_DENY

Secondary Symptom: Intermittent routing table visibility

Root Cause: Missing or incorrect contract filter

ELAM Proves: Route exists and was hit

Do NOT Troubleshoot: Routing / L3Out

Rule: If FIB hit + policy deny → policy always wins.

Scenario 2 – Route Exists but Still Drops

================= Forwarding Lookup (FPB) =================

Dst IP Lookup VRF: 0x20004

Dst IP Hit: YES

Dst IP Hit Index: 0x6100

NextHop L2 Ptr Valid: NO

================= MAC Lookup =================

Dst MAC Hit: NO

================= Rewrite / Drop =================

Lookup Drop Reason: NO_ADJ

Extra Info

“BGP is up and routes look fine on APIC.”

Adjacency Beats Routing

Primary Drop: NO_ADJ

Routing Not Issue Because: Route hit is confirmed

Root Cause: ARP / ND unresolved

Verify Next: ARP table, endpoint reachability

Do NOT Blame: Routing protocol team

Rule: Route without MAC = dead end.

Scenario 3 – VLAN vs Port Security Confusion [Nexus OS]

================= Captured Packet =================

Src MAC: 00:aa:bb:cc:dd:ee

Ingress VLAN: 200

================= Forwarding Lookup =================

VLAN Translation: HIT

================= MAC Lookup =================

Src MAC Hit: NO

Src MAC Learn Allowed: NO

================= Rewrite / Drop =================

Lookup Drop Reason: SMAC_MISS

Extra Info

“The VLAN is allowed on the port.”

Port Security Masquerading as VLAN Issue

Primary Drop: SMAC_MISS

VLAN Not Issue Because: VLAN Translation HIT

Root Cause: MAC limit exceeded

Check: Port security / MAC limit on EPG

Decisive Field: Src MAC Learn Allowed: NO

Rule: Ingress succeeded; learning failed.

Scenario 4 – Contract Present but Traffic Still Fails

================= Forwarding Lookup (FPB) =================

Dst IP Lookup VRF: 0x20003

Dst IP Hit: YES

================= Contract Lookup (FPC) =================

Sclass: 16385

Dclass: 16390

Contract Result: DROP

SECURITY_GROUP_DENY: YES

Extra Info

“A contract is already configured between the EPGs.”

Contract Exists ≠ Contract Permits

Why Dropped: Filter missing / wrong direction / wrong ports

Likely Missing: L4 port or protocol

ELAM Confirms: Exact deny at FPC stage

Next Command: show system internal aclqos zoning-rules

Rule: Contracts are precise, not implicit.

Scenario 5 – Multiple Drops Seen in Fabric

Leaf-101 ELAM:

Lookup Drop Reason: NO_ADJ

Leaf-102 ELAM:

Lookup Drop Reason: UC_PC_CFG_TABLE_DROP

Extra Info

“Traffic is between two endpoints in different pods.”

Scenario 5 — Downstream vs Upstream Trap

Primary Drop: UC_PC_CFG_TABLE_DROP (Leaf-102)

Secondary Drop: NO_ADJ (Leaf-101)

Real Problem: Leaf-102 missing route

Why Both Appear: Leaf-101 forwards toward Leaf-102, which drops

Fix: Leaf-102 routing

Rule: Fix the first real drop in the path.

---------------------------------------------------------------------------------

ACI ELAM Trigger Examples

1. VLAN Bridging (L2 Switching )

Same BD / same subnet traffic

vsh_lc

debug platform internal tah/roc/app elam asic 0

trigger reset

trigger init in-select 6 out-select 1

set outer l2 vlan 101

set outer l2 src_mac 00:11:22:33:44:55

set outer l2 dst_mac aa:bb:cc:dd:ee:ff

start

status

ereport

2. L3 Routed IPv4 Traffic (Inter-EPG / Inter-Subnet)

Most common TAC use case

vsh_lc

debug platform internal tah/roc/app elam asic 0

trigger reset

trigger init in-select 6 out-select 1

set outer ipv4 src_ip 192.168.1.10

set outer ipv4 dst_ip 10.10.10.20

set outer ipv4 protocol 6

set outer l4 dst_port 443

start

status

report detail

ereport

3. VXLAN Encapsulated Traffic (Fabric / Overlay)

Run on spine, egress leaf, or border leaf

trigger reset

trigger init in-select 14 out-select 1

set outer udp dst_port 4789

set outer vxlan vnid 123456

set inner ipv4 src_ip 192.168.100.1

set inner ipv4 dst_ip 192.168.200.1

start

status

report detail

ereport

4. VMM Traffic (VM-to-VM / VM-to-Baremetal)

Still endpoint ingress → treated as VLAN traffic

trigger reset

trigger init in-select 6 out-select 1

set outer l2 vlan 102

set outer l2 src_mac 00:50:56:aa:bb:cc

set outer l2 dst_mac 00:50:56:dd:ee:ff

start

status

report detail

ereport

5. ICMP (Ping) Traffic

Quick routing vs policy validation

trigger reset

trigger init in-select 6 out-select 1

set outer ipv4 src_ip 10.1.1.1

set outer ipv4 dst_ip 10.1.2.2

set outer ipv4 protocol 1

start

status

report detail

ereport