NTP in ACI

NTP uses the User Datagram Protocol (UDP) & all NTP communications use Coordinated Universal Time (UTC).

NTP uses a Stratum to describe the distance between a network device and an authoritative time source. A stratum 1 time server is directly attached to an authoritative time source (such as a radio or atomic clock or a GPS time source). A Stratum 2 NTP server receives its time through NTP from a stratum 1 time server.

Configure Out-of-Band Management

Step 1: Go to the Tenants menu, select the mgmt tenant, navigate to Node Management Addresses > Static Node Management Addresses and right-click the menu to Create Static Node Management Addresses.

Step 2 : Configure the OOB management addresses for nodes 101–102 (leaf-a and leaf-b) with the settings below. Then click Submit and Yes to confirm the configuration.

Node Range: 101-102

Config: Select Out-Of-Band Addresses

Out-Of-Band management EPG: default

Out-Of-Band IPv4 address (include the mask): 192.168.10.211/24

Out-Of-Band IPv4 gateway: 192.168.10.254

Note: The defined IP address is assigned to node 101, the next incremented IP address is assigned to node 102.

Similarly Configure the OOB management address for node 201 (spine) with the settings below. Then click Submit and Yes to confirm the configuration.

Node Range: 201-201

Config: Select Out-Of-Band Addresses

Out-Of-Band management EPG: default

Out-Of-Band IPv4 address (include the mask): 192.168.10.213/24

Out-Of-Band IPv4 gateway: 192.168.10.254

Step 3: Verify if management ip address is properly configured

GUI

Fabric > Inventory > Pod 1, select a switch and choose the General tab.

You can also examine the OOB management IP address from Inventory by expanding the switch and selecting Interfaces > Management Interfaces > mgmt0

leaf-a# show vrf

VRF-Name VRF-ID State Reason

black-hole 3 Up --

management 2 Up --

overlay-1 4 Up --

leaf-a# show ip interface brief vrf management

IP Interface Status for VRF "management"(2)

Interface Address Interface Status

mgmt0 192.168.10.211/24 protocol-up/link-up/admin-up

Let's Configure NTP

The NTP configuration will be applied to the fabric.

Step1: Date and Time Policy

Fabric > Fabric Policies > Policies > Pod > Date and Time. Right-click the menu and select Create Date and Time Policy.

Step2: Name the policy NTP, leave the states unchanged (administrative state enabled, server state disabled, authentication state disabled) and click Next.

Click the plus sign (+) in the table to add an NTP server with the settings below. Leave other settings at their default values, then click OK and Finish.

Name: 192.168.10.15 (IP address of your StudentPC, which is running an NTP daemon)

Management EPG: default (Out-of-Band).

Step3: In Fabric > Fabric Policies > Pods, right-click Policy Groups to Create Pod Policy Group.

Step 4: Configure the pod policy group name Pod_PG and choose the Date Time policy NTP. Click Submit.

Step 5: Fabric > Fabric Policies > Pods > Profiles > Pod Profile default > default. Choose the pod selector default and select the Pod_PG policy from the Fabric Policy Group drop-down. Click Update and Continue.

Step 6: Fabric > Fabric Policies > Policies > Pod > Date and Time > Policy NTP > NTP Server 192.168.10.15, select the Operational tab and examine the Sync Status.

If there are multiple NTP providers, flag at least one of them as the preferred time

source using the 'Preferred' checkbox as per the figure below.

apic# ntpstat

synchronised to NTP server (192.168.10.15) at stratum 14

time correct to within 16 ms

polling server every 16 s

apic# show ntpq

nodeid remote refid st t when poll reach

auth delay offset jitter

------ - ------------------------------ -------------------------- -------- -- -------- -------- -------

- ---- -------- -------- --------

1 * 10.48.37.151 173.38.201.115 2 u 25 64 377

none 0.214 -0.118 0.025

2 * 10.48.37.151 173.38.201.115 2 u 62 64 377

none 0.207 -0.085 0.043

3 * 10.48.37.151 173.38.201.115 2 u 43 64 377

none 0.109 -0.072 0.030

apic# show clock

Time : 17:38:05.814 UTC Wed Oct 02 2019

leaf# show ntp peers

-----------------------------------------------------------------------------

Peer IP Address Serv/Peer Prefer KeyId Vrf

-----------------------------------------------------------------------------

10.48.37.151 Server yes None management

leaf1# show ntp peer-status

Total peers : 1

* - selected for sync, + - peer mode(active),

- - peer mode(passive), = - polled in client mode

remote local st poll reach delay vrf

--------------------------------------------------------------------------------

*10.48.37.151 0.0.0.0 2 64 377 0.000 management

Note: If the NTP server was the preferred one and was deleted. This could result in a ntp sync issue.

Please add back it as preferred server

The devices will sync to it … remove it as preferred ntp server and remove it again.

1. Copy the current ntp configuration to ntp.conf.dhcp

2. cp /etc/ntp.conf /var/lib/ntp/ntp.conf.dhcp

3. Restart NTP service (systemctl restart ntp)

4. Disable and enable ntp feature on leaf switch.

Logs for NTP

 /var/sysmgr/tmp_logs/ntpwd.log

/var/sysmgr/mem_logs/ntpd_logs

leaf# show ntp (ESC) (ESC) authentication-keys internal peers statistics authentication-status peer-status server-info trusted-keys

leaf# show ntp peer-status Total peers : 0 * - selected for sync, + - peer mode(active), - - peer mode(passive), = - polled in client mode remote local st poll reach delay vrf

If there is no connection to the NTP server, whether through out-of-band or in-band, the switch will not be able to provide NTP peer status information.

leaf1# show ntp statistics peer ipaddr 10.48.37.151

...

packets sent: 256

packets received: 256

leaf1# tcpdump -i eth0 udp port 161

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

22:18:10.204011 IP 10.155.0.153.63392 > 10.48.22.77.snmp: C=my-snmp-community GetNextRequest(28)

.iso.0.8802.1.1.2.1.1.1.0

22:18:10.204558 IP 10.48.22.77.snmp > 10.155.0.153.63392: C=my-snmp-community GetResponse(29)

.iso.0.8802.1.1.2.1.1.2.0=4

spine1# show bgp vpnv4 unicast summary vrf overlay-1

BGP summary information for VRF overlay-1, address family VPNv4 Unicast

BGP router identifier 10.0.136.65, local AS number 65001

BGP table version is 15, VPNv4 Unicast config peers 7, capable peers 6

0 network entries and 0 paths using 0 bytes of memory

BGP attribute entries [0/0], BGP AS path entries [0/0]

BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

10.0.136.64 4 65001 162 156 15 0 0 02:26:00 0

10.0.136.67 4 65001 154 154 15 0 0 02:26:01 0

10.0.136.68 4 65001 152 154 15 0 0 02:26:00 0

10.0.136.69 4 65001 154 154 15 0 0 02:26:01 0

10.0.136.70 4 65001 154 154 15 0 0 02:26:00 0

10.0.136.71 4 65001 154 154 15 0 0 02:26:01 0

spine1# show bgp vpnv6 unicast summary vrf overlay-1

BGP summary information for VRF overlay-1, address family VPNv6 Unicast

BGP router identifier 10.0.136.65, local AS number 65001

BGP table version is 15, VPNv6 Unicast config peers 7, capable peers 6

0 network entries and 0 paths using 0 bytes of memory

BGP attribute entries [0/0], BGP AS path entries [0/0]

BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

10.0.136.64 4 65001 162 156 15 0 0 02:26:11 0

10.0.136.67 4 65001 155 155 15 0 0 02:26:12 0

10.0.136.68 4 65001 153 155 15 0 0 02:26:11 0

10.0.136.69 4 65001 155 155 15 0 0 02:26:12 0

10.0.136.70 4 65001 155 155 15 0 0 02:26:11 0

10.0.136.71 4 65001 155 155 15 0 0 02:26:12 0