TCPDUMP

This is a very useful tool for capturing traffic destined for the leaf, spine or APIC i.e. within fabric.

This method can be used also for traffic reaching CPU (Ping to/from the switch, ARP of pervasive gateway, NTP, SNMP)

LEAF/SPINE

Interface to be used:

1) inband traffic use kpm_inb

2) For OOB traffic use eth0

Note: ARP for kpm_inb only shows traffic in one direction

leaf1# tcpdump -D (Specify the available interfaces to capture on)

Icmp (capture only ping packets)

3) leaf# tcpdump -D

1.eth0 For OOB traffic

2.inband_lo

3.inband_hi

4.mgmt0

5.psdev0

6.kpm_inb for most inband traffic

7.kpm_mgmt

8.tahoe0

9.psdev2

10.any (Pseudo-device that captures on all interfaces)

11.lo

4) leaf# tcpdump -i eth0

5) leaf# tcpdump -i eth0 -f icmp

5) leaf# tcpdump -i kpm_inb

6) leaf# tcpdump -i kpm_inb icmp

7) Writing output to a file

leaf# tcpdump -i kpm_inb icmp -w /tmp/capture.pcap

8)Reading output from a capture file

leaf# tcpdump -r /tmp/capture.pcap

9)Reading the output without resolving the names, displaying only IPs, using “-n”

leaf1# tcpdump -n -r /tmp/capture.pcap

10) tcpdump for L2 host

Leaf # tcpdump -i kpm_inb ether host 00:25:b5:dd:c1:12 and icmp -v

This command uses tcpdump to capture and display network traffic in real time with the following specifics:

• Interface Selection (-i kpm_inb):The command monitors traffic on the network interface named kpm_inb.

• Filtering by Ethernet Host (ether host 00:25:b5:dd:c1:12):It captures packets where the specified MAC address (00:25:b5:dd:c1:12) appears either as the source or destination. This is useful for isolating traffic related to a particular device.

• Filtering by Protocol (and icmp):Only ICMP packets are captured. ICMP is typically used for diagnostic or error messages (for example, ping requests and replies).

• Verbose Output (-v):The -v flag makes the output more detailed, providing additional information about each captured packet.

11) Adding host filter

leaf# tcpdump -i any icmp and src host 192.168.3.254

leaf# tcpdump -i any icmp and dst host 192.168.3.254

12) Adding src/dst host filter

leaf# tcpdump -i eth0 icmp and src host 192.168.3.254

leaf# tcpdump -i eth0 icmp and dst host 192.168.3.254

leaf# tcpdump -i kpm_inb icmp and src host 192.168.3.254

leaf# tcpdump -i kpm_inb icmp and dst host 192.168.3.254

13) Adding verbose option

leaf# tcpdump -vvi kpm_inb icmp and host 192.168.3.254

14) tcpdump on knet or tahoe0 interface

knet interface on Gen-1 and tahoe0 interface on Gen-2 Hardware show all packet but they have an internal header so not fully readable.

In Gen-2 Hardware you can use tcpdump2 which is script decoding internal header on the top of tcpdump.

leaf1# tcpdump2 -i tahoe0  host 1.1.1.1

15) Tcpdump on basis of port

leaf# tcpdump -i eth0 -f port 162 -vv

leaf# tcpdump -i any port 179

16) To capture TCP port 25 traffic specifically from 192.168.3.254 to 192.168.3.3

tcpdump -i any tcp port 25 and src host 192.168.3.254 and dst host 192.168.3.3

tcpdump -i any tcp port 25 and src host 192.168.3.254

tcpdump -i any tcp port 25

Tcpdump on APIC

Need root access

Command syntax:

[root@apic1 ~]# tcpdump -h (help)

[root@apic1 ~]# tcpdump -D     

1.bond0      > Traffic from APIC-to-APIC and APIC-to-switches (inband)

2.oobmgmt > Packets entering/leaving via the APIC out-of-band management interface.

3.bond0.3932

4.lxcbr0

5.ifb0

6.docker0

7.tep0

8.bond0.101

9.nflog (Linux netfilter log (NFLOG) interface)

10.eth1-1

11.eth1-2

12.bond1

13.tep1

14.teplo-1

15.usbmon1 (USB bus number 1)

16.eth2-1

17.eth2-2

18.tep2

19.usbmon2 (USB bus number 2)

20.tep3

21.tep4

22.tep5

23.tep6

24.tep7

25.veth3866601

26.any (Pseudo-device that captures on all interfaces)

27.lo [Loopback]

tcpdump on oobmgmt, with icmp toggle

1) tcpdump on oobmgmt

[root@apic1 ~]# tcpdump -i oobmgmt

[root@apic1 ~]# tcpdump -i oobmgmt icmp

2) tcpdump on port

[root@apic1 ~]# tcpdump -i oobmgmt port 22

3) tcpdump on bond0, with a host filter

[root@apic1 ~]# tcpdump -i bond0 host spine1

BGP Traffic

tcpdump -i any port 179 -n | grep 3.6.6.9 -w /tmp/bgp_traffic.pcap

• tcpdump → Packet capture tool.

• -i any → Captures packets on all available interfaces.

• port 179 → Filters only BGP (Border Gateway Protocol) traffic, as BGP uses TCP port 179.

• -n → Prevents hostname and service resolution, displaying raw IP addresses.

• | grep 3.6.6.9 → Filters output to show only packets involving the specified IPv4 address.

To capture BGP traffic in ACI L3out

Leaf# tcpdump -i kpm_inb port 179 and host 10.155.7.42 and host 10.156.3.122

OSPF Traffic

On ACI leaf nodes, the user can perform tcpdump on the 'kpm_inb' CPU interface to

verify if the protocol packets have reached the leaf CPU. 

IP Protocol Number: proto 89 (IPv4) or ip6 proto 0x59 (IPv6)

OSPF Link Local Mcast IP: host 224.0.0.5 or host 224.0.0.6

leaf# tcpdump -ni kpm_inb proto 89

Leaf # tcpdump -ni kpm_inb proto 0x59

EIGRP Traffic

Users can execute tcpdump on the 'kpm_inb' CPU interface of ACI leaf nodes to

verify if protocol packets have reached the leaf's CPU. Apply IP protocol number 88

(EIGRP) as a filter.

leaf# tcpdump -ni kpm_inb proto 88

DHCP Traffic

leaf # tcpdump -i any -f port 67 or port 68 -w /tmp/dhcp_capture.pcap

leaf # tcpdump -i any -n -vvv -s 0 -l port 67 or port 68

• -n: Don't resolve names (faster, cleaner output).

• -vvv: Very verbose (shows full DHCP details).

• -s 0: Capture full packet.

• -l: Line-buffered output for real-time viewing.

leaf # tcpdump -i any -n portrange 67-68 -w /tmp/dhcp_capture.pcap

-n: Don't resolve hostnames.

DNS Traffic

leaf# tcpdump -i any port 53 -w /tmp/capture.pcap

Note : Port 53 captures both DNS queries and responses, which might generate high traffic.

To capture ipv6 traffic

Leaf # tcpdump -i any ipv6 host 2405:200:a70:fa30::5 -w /tmp/capture.pcap

PING

Leaf # iping6 -c 5 2405:200:a70:fa30::5

Leaf # tcpdump -i any dst host 107.191.249.85 and src host 10.217.189.37 -w /tmp/capture.pcap

Leaf # tcpdump -i any tcp and host 106.221.209.25 tcpdump -i any ip and src host 27.59.108.38

List the Top 100 Largest Files/Directories in Current Directory

ls -lSh | head -100

• ls -lSh: Lists items by size in human-readable format, sorted descending.

• head -100: Shows the top 100 largest items, including directories.

ls -Sh | head -100 | xargs rm -f

• ls -Sh lists items by size in descending order, but does not use a long-list format.

• head -100 picks the top 100.

• rm -f removes files without prompt

• It will fail on directories but won’t remove them

Step 1: See What Will Be Deleted

ls -lSh | head -100 | awk '{print $NF}'

• ls -lSh: Long-list all items (files + dirs) in human-readable format, sorted by size descending.

• head -100: Take the top 100 lines.

• awk '{print $NF}': Extracts the last field, which should be the name of the file/directory.

Review the output carefully to make sure you are okay deleting them.

Step 2: Actually Delete Them

ls -lSh | head -100 | awk '{print $NF}' | xargs -I{} rm -rf {}

xargs -I{} rm -rf {}: For each item in the list, recursively remove it. This removes files or directories.

Find Command Examples

1. List Files Matching Certain Criteria

   
   

   # Display names of heapdump files older than 90 days

   
   

   find /data2/infra/elastic -type f -name "heapdump*" -mtime +90 -print

   
   

   # Display details (permissions, size, etc.) of heapdump files older than 90 days

   
   

   find /data2/infra/elastic -type f -name "heapdump*" -mtime +90 -ls

2. Find & Delete Files

   
   

   # Delete heapdump files older than 90 days

   
   

   find /data2/infra/elastic -type f -name "heapdump*" -mtime +90 -exec rm -f {} \;

   
   

   • find /data2/infra/elastic: Searches in /data2/infra/elastic

   • -type f: Limits the search to files

   • -name "heapdump"*: Matches files beginning with heapdump

   • -mtime +90: Filters files older than 90 days

   • -exec rm -f {} ;: Executes a forced remove on each matching file

     
     

3. Delete with Confirmation

   
   

   # Prompt before deleting each file

   
   

   find /data2/infra/elastic -type f -name "heapdump*" -mtime +90 -ok rm -f {} \;

4. Log Deleted Files

# Remove heapdump files older than 90 days and log each deletion

find /data2/infra/elastic -type f -name "heapdump*" -mtime +90 -exec rm -f {} \; -exec echo "Deleted: {}" >> /var/log/heapdump_cleanup.log \;

5. Move Instead of Delete

find /data2/infra/elastic -type f -name "heapdump*" -mtime +90 -exec mv {} /backup/heapdumps/ \;

6. Find Files Over 90 Days Old in directory "/data/log" – Dry Run

   find /data/log -type f -mtime +90 -exec echo rm -f {} \;

   
   

   • -type f: Only matches regular files.

   • -mtime +90: Last modified more than 90 days ago.

   • -exec echo rm -f {} ;: Shows what would be deleted, but does not delete anything (safe to review).

     
     

7. Find Files Over 90 Days Old in /data/log – Actual Deletion

   find /data/log -type f -mtime +90 -exec rm -f {} \;

   
   

8. List the Top 100 Largest Files (Not Directories) in Current Directory

   find . -maxdepth 1 -type f -exec ls -Sh {} + 2>/dev/null | head -100

   • -maxdepth 1: Only look in the current directory (no subdirectories).

   • -type f: Matches files only (no directories).

   • -exec ls -Sh {} +: Lists matched files sorted by size.

   • head -100: Shows the top 100 largest files.

     
     

9. Delete the Top 100 Largest Files (Not Directories) in Current Directory

   find . -maxdepth 1 -type f -exec ls -Sh {} + 2>/dev/null | head -100 | awk '{print $NF}' | xargs -d '\n' rm -f

   • Same logic as above, but piped into awk and rm to perform deletion.

   • awk '{print $NF}': Extracts the filename.

   • xargs -d '\n' rm -f: Removes each file listed.

Managed Object Query

APIC CLI moquery Commands

MO Class	Command	Description
fvTenant	moquery -c fvTenant	Lists all tenant configurations in the fabric.
fvCtx	moquery -c fvCtx	Shows VRF (context) objects that define routing domains.
fvBD	moquery -c fvBD	Displays Bridge Domain objects, which represent L2 forwarding domains.
fvSubnet	moquery -c fvSubnet	Lists subnets defined under a Bridge Domain.
fvAp	moquery -c fvAp	Shows Application Profile objects that group related EPGs.
fvAEPg	moquery -c fvAEPg	Lists Endpoint Groups (EPGs) along with their associated attributes.
fvCEp	moquery -c fvCEp	Returns the endpoints (IP/MAC pairs) learned by the fabric.
l3extOut	moquery -c l3extOut	Displays L3Out objects that configure external (routed) connectivity.
l3extInstP	moquery -c l3extInstP	Shows L3 external instance profiles associated with L3Outs.
l3extSubnet	moquery -c l3extSubnet	Lists subnets defined for external routing and connectivity.
fabricNode	moquery -c fabricNode	Retrieves information about fabric nodes (APIC, spine, and leaf details).
fabricPod	moquery -c fabricPod	Lists pod-level information in multi-pod deployments.
infraAccPortP	moquery -c infraAccPortP	Lists all interface policies for physical ports.
infraAccPortGrp	moquery -c infraAccPortGrp	Shows interface policy groups associated with access ports.
infraAccBndlGrp	moquery -c infraAccBndlGrp	Displays port channel (vPC) interface policy groups.
vlanCktEp	moquery -c vlanCktEp	Lists VLAN encapsulation information for endpoints.
fvRsPathAtt	moquery -c fvRsPathAtt	Displays port paths associated with an EPG.
topSystem	moquery -c topSystem	Shows system information about APIC and switches.
vzFilter	moquery -c vzFilter	Lists contract filters used for communication between EPGs.
vzBrCP	moquery -c vzBrCP	Displays contract policies applied between EPGs.
vzSubj	moquery -c vzSubj	Shows contract subjects that define allowed communication.
mgmtMgmtP	moquery -c mgmtMgmtP	Lists out-of-band (OOB) and in-band management policies.

Switch CLI moquery Commands (via VSH)

MO Class	Command	Description
fvCEp	moquery -c fvCEp	Lists endpoints that the switch has learned directly from attached devices.
fvAEPg	moquery -c fvAEPg	Displays the EPGs present on the switch (reflects the broader fabric but with local context).
fvBD / fvSubnet	moquery -c fvBD   moquery -c fvSubnet	Shows local Bridge Domain and subnet details as the switch understands them.
fabricNode / fabricPod	moquery -c fabricNode   moquery -c fabricPod	Provides details on the switch’s role in the fabric and the pod configuration.
topSystem	moquery -c topSystem	Shows system information about the local switch.
ethpmPhysIf	moquery -c ethpmPhysIf	Displays physical interface information, including state and errors.
eqptIngrTotal5min	moquery -c eqptIngrTotal5min	Retrieves ingress traffic statistics on a per-interface basis.
eqptEgrTotal5min	moquery -c eqptEgrTotal5min	Retrieves egress traffic statistics per interface.
eqptFan	moquery -c eqptFan	Shows fan status for the switch.
eqptPsu	moquery -c eqptPsu	Displays power supply details for the switch.

SPINE# moquery -c coopIpv4Rec -f 'coop.Ipv4Rec.addr=="10.88.156.30"'

This command is used in Cisco ACI (Application Centric Infrastructure) to query the COOP (Council of Oracles Protocol) database for a specific IPv4 record.

APIC# moquery -c epmIpEp -f 'epm.IpEp.addr=="10.88.156.30"' | egrep "dn|ifId|pcTag"

Breaking Down the Command:

1. moquery -c epmIpEp

   • Queries the EPG (Endpoint Manager) database for IPv4 endpoint (epmIpEp).

   • This class stores endpoint information learned by leaf switches.

2. -f 'epm.IpEp.addr=="10.88.156.30"'

   • Filters the output for the specific IP address 10.88.156.30.

3. | egrep "dn|ifId|pcTag"

   • Uses egrep (extended grep) to display only lines containing:

     • dn → Distinguished Name (identifies the location of the endpoint in ACI fabric).

     • ifId → Interface ID (which physical or logical interface the endpoint was learned from).

     • pcTag → Policy Control Tag, used to enforce policies in ACI.

   
   

   Expected Output Example:

   If the endpoint is found in the fabric, you may see something like this:

   
   

   dn: topology/pod-1/node-102/sys/ctx-[vxlan-12345]/bd-[vxlan-67890]/ip [10.88.156.30]

   ifId: eth1/3

   pcTag: 32768

Interpreting the Output:

• dn:

  • topology/pod-1/node-102 → The Pod and Node ID where the endpoint is learned.

  • ctx-[vxlan-12345] → The VRF (Context) associated with the endpoint.

  • bd-[vxlan-67890] → The Bridge Domain where the endpoint resides.

  • ip-[10.88.156.30] → The queried IP address.

• ifId: eth1/3

  • Indicates the physical interface on the leaf switch where the endpoint is learned.

• pcTag: 32768

  • ACI Policy Control Tag (pcTag) used to define policies.

  • 32768 is the default tag for learned endpoints.

Additional Notes:

• Use moquery -c <MO-Class> -x to get more detailed output, including object attributes.

• Pipe the output with grep for focused searches, e.g., moquery -c fvCEp | grep 10.1.1.1.

• Some moquery commands work only on APICs, while others return relevant switch-local information.