Original document with latest updates:
• Part1: https://wiki.cnap.hv.se/practice/ccnp/labs/evpn-noFlood-1
• Part2: https://wiki.cnap.hv.se/practice/ccnp/labs/evpn-noFlood-2
• Part3: https://wiki.cnap.hv.se/practice/ccnp/labs/evpn-noFlood-3
This 3-part-lab is only configuring Layer2 VXLAN; Anycast Gateway, VRF & Routing is a completely different 🫔enchilada.
¶ Part1: Create the Underlay; Check Connectivity
• You Are Here --> Part2: Tunnel --> Part3: MP-BGP
According to RFC 7348: https://datatracker.ietf.org/doc/html/rfc7348
Virtual eXtensible Local Area Network (VXLAN):
A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks
- Debugging: troubleshoot-evpn-vxlanv6-on-catalyst-90
- Cisco reference page (Catalyst 9K): implement-bgp-evpn-routing-policy-on-catalyst
- Another Cisco Doc: nexus-9000-series-switches/guide
- Fortigate: administration-guide: vxlan-troubleshooting
¶ Create Physical Underlay
Use three Multi-Layer Switches (MLS, not L2), use black 3650 as spine and 9300L as leafs.
Connect 4 cables according to topology and formula: Cable X goes to port X. Example 3:rd cable goes to port 3 on both ends.
Leaf 1, port 1 --> Spine 3, port 1
Leaf 1, port 2 --> Spine 3, port 2
Leaf 2, port 3 --> Spine 3, port 3
Leaf 2, port 4 --> Spine 3, port 4
Spine-3#show cdp neighbors
Device ID Local Intrfce Holdtme Capability Platform Port ID
Leaf-1 Gig 1/0/1 173 R S I C9300L-24 Gig 1/0/1
Leaf-1 Gig 1/0/2 135 R S I C9300L-24 Gig 1/0/2
Leaf-2 Gig 1/0/3 126 R S I C9300L-24 Gig 1/0/3
Leaf-2 Gig 1/0/4 134 R S I C9300L-24 Gig 1/0/4
¶ Create Routed ports, turn off spanning tree
On all 3 units:
Please use Copy button in the top right corner of the below black box.
no ip domain lookup
interface range gi 1/0/1 - 9
no switchport
no spanning-tree vlan 1-4094
¶ Create loopbacks and OSPF underlay
- Create one subnet for each spine-leaf cable
- Create OSPF underlay for full connectivity between Routed-Ports
Leaf-1:
hostname Leaf-1
ip routing
interface loopback 1
ip address 10.1.1.1 255.255.255.255
interface gi 1/0/1
ip address 10.0.1.1 255.255.255.0
interface gi 1/0/2
ip address 10.0.2.1 255.255.255.0
router ospf 1
network 10.0.0.0 0.255.255.255 area 0
Leaf-2:
hostname Leaf-2
ip routing
interface loopback 2
ip address 10.2.2.2 255.255.255.255
interface gi 1/0/3
ip address 10.0.3.2 255.255.255.0
interface gi 1/0/4
ip address 10.0.4.2 255.255.255.0
router ospf 1
network 10.0.0.0 0.255.255.255 area 0
Spine-3:
! This is the old, cool, black c3650
hostname Spine-3
ip routing
interface loopback 3
description CP - Control Plane
ip address 10.3.3.3 255.255.255.255
interface loopback 4
description L2DP - Layer 2 Data Plane
ip address 10.4.4.4 255.255.255.255
interface gi 1/0/1
ip address 10.0.1.3 255.255.255.0
interface gi 1/0/2
ip address 10.0.2.3 255.255.255.0
interface gi 1/0/3
ip address 10.0.3.3 255.255.255.0
interface gi 1/0/4
ip address 10.0.4.3 255.255.255.0
router ospf 1
network 10.0.0.0 0.255.255.255 area 0
Loopback 3 (Control Plane): This is used as the BGP Peering IP. It establishes the BGP EVPN neighbor relationships to exchange MAC/IP route updates.
Loopback 4 (Data Plane): This is configured as the VTEP Source IP.
¶ Resulting configuration
See black boxes above (use "copy" on the top right corner to cheat 
)
¶ 
Verify ROUTES in OSPF
Leaf-1#
show ip route
... Gateway of last resort is not set 10.0.0.0/8 is variably subnetted, 9 subnets, 2 masks C 10.0.1.0/24 is directly connected, GigabitEthernet1/0/1 L 10.0.1.1/32 is directly connected, GigabitEthernet1/0/1 C 10.0.2.0/24 is directly connected, GigabitEthernet1/0/2 L 10.0.2.1/32 is directly connected, GigabitEthernet1/0/2 O 10.0.3.0/24 [110/2] via 10.0.2.3, 00:00:05, GigabitEthernet1/0/2 [110/2] via 10.0.1.3, 00:00:10, GigabitEthernet1/0/1 O 10.0.4.0/24 [110/2] via 10.0.2.3, 00:00:05, GigabitEthernet1/0/2 [110/2] via 10.0.1.3, 00:00:10, GigabitEthernet1/0/1 C 10.1.1.1/32 is directly connected, Loopback1 O 10.2.2.2/32 [110/3] via 10.0.2.3, 00:00:05, GigabitEthernet1/0/2 [110/3] via 10.0.1.3, 00:00:10, GigabitEthernet1/0/1 O 10.3.3.3/32 [110/2] via 10.0.2.3, 00:00:05, GigabitEthernet1/0/2 [110/2] via 10.0.1.3, 00:00:10, GigabitEthernet1/0/1
Look!! We have sometimes two (2) lines (paths) for one (1) network; that is Equal Cost Muliple PAths (ECMP) load balancing by Cisco Express Forwarding (CEF). This is done automatically, no extra config in OSPF 
Note that 10.0.y.z networks are links between devices and that 10.x.x.x are loopback of devices
¶ Verify Underlay connectivity
Underlay works
Leaf-1
ping 10.2.2.2 source 10.1.1.1
Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.2.2.2, timeout is 2 seconds: Packet sent with a source address of 10.1.1.1 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
Leaf-1
trace 10.2.2.2 source 10.1.1.1
Type escape sequence to abort. Tracing the route to 10.2.2.2 VRF info: (vrf in name/id, vrf out name/id) 1 10.0.1.3 2 msec 10.0.2.3 2 msec 10.0.1.3 2 msec 2 10.0.4.2 2 msec * 10.0.3.2 2 msec Leaf-1#
Look, two lines but only one "1"; that means ECMP load-sharing over two paths
¶ Roadmap
• You Are Here --> Part2: Tunnel --> Part3: MP-BGP