BGP (Border Gateway Protocol) is the routing protocol of the internet. Understanding BGP is essential for large-scale infrastructure and cloud networking.
Why BGP?
Simple routing (RIP, OSPF):
- Works for small networks
- Scales to hundreds of nodes
- High overhead, convergence time seconds
BGP:
- Works at internet scale (hundreds of thousands of routes)
- Designed for autonomous systems
- Allows policy-based routing
- Convergence time minutes
Autonomous System (AS)
Concept: Independent network authority
AS 65001 (Company A)
├─ Network 10.0.0.0/16
├─ Network 10.1.0.0/16
└─ Network 10.2.0.0/16
AS 65002 (Company B)
├─ Network 172.16.0.0/16
├─ Network 172.17.0.0/16
└─ Network 172.18.0.0/16
BGP: "AS 65001 can reach these networks"
ASN (Autonomous System Number):
- 16-bit: 1-65535 (original)
- 32-bit: 1-4294967295 (modern)
- Private ASNs: 64512-65534
BGP Basics
Two Types of BGP
eBGP (External BGP) Communication between different AS numbers:
Company A (AS 65001) ← eBGP ← ISP (AS 4128)
iBGP (Internal BGP) Communication within same AS:
Router A (AS 65001) ← iBGP ← Router B (AS 65001)
BGP Routing Decisions
BGP doesn't just use shortest path like OSPF. It uses multiple attributes:
Path Attributes (Decision Order):
-
Weight (Cisco proprietary)
- Higher wins
- Local preference only
-
Local Preference
- Higher wins
- "Prefer this exit point"
-
Originated Locally
- Routes originated locally preferred over learned routes
-
AS Path Length
- Shorter wins
- "Fewer hops"
-
Origin
- IGP > EGP > Incomplete
-
MED (Multi-Exit Discriminator)
- Lower wins
- "Use this entry point"
-
eBGP vs iBGP
- External preferred
-
IGP Metric
- To reach next-hop
-
Tiebreaker
- Oldest route
- BGP router ID
BGP Route Advertisement
Network Advertisement:
# BGP configuration (Cisco example)
router bgp 65001
neighbor 203.0.113.1 remote-as 64512 # ISP's BGP
network 10.0.0.0 mask 255.255.0.0 # Advertise this
network 10.1.0.0 mask 255.255.255.0 # And thisWhat gets advertised:
AS 65001 announces:
"I can reach 10.0.0.0/16"
"I can reach 10.1.0.0/16"
AS 64512 (ISP) receives announcements
Tells other networks: "You can reach these via me"
BGP Route Aggregation
Multiple specific routes → Single aggregate:
Without aggregation:
10.0.0.0/24 (advertise)
10.0.1.0/24 (advertise)
10.0.2.0/24 (advertise)
10.0.3.0/24 (advertise)
With aggregation:
10.0.0.0/22 (covers all four)
Smaller routing table, less table churn, faster convergence
BGP in Cloud Networking
AWS Direct Connect + BGP:
┌──────────────┐
│ AWS Account │
├──────────────┤
│ VPC │
│ 10.0.0.0/8 │
├──────────────┤
│ AWS BGP │
│ AS 14061 │
└────────┬─────┘
│ BGP Session
↓
┌────────────────────┐
│ Customer BGP Router│
│ Advertises on-prem │
→ routes │
└────────────────────┘
↑
│ On-Prem network
↓
┌────────────────────┐
│ Your Data Center │
│ 172.16.0.0/12 │
└────────────────────┘
BGP Session:
AWS: "VPC at 10.0.0.0/8"
Customer: "On-prem at 172.16.0.0/12"
Traffic flows through Direct Connect
GCP Cloud Router + BGP:
Similar behavior, uses BGP to share routes
Between VPCs and on-premise networks
BGP Policy Routing
BGP allows sophisticated policies:
Example: Prefer one ISP for outbound, another for inbound
Outbound (Local Preference):
- ISP-1: local-preference 200 (preferred)
- ISP-2: local-preference 100
Inbound (MED):
- ISP-1: MED 50
- ISP-2: MED 100
Result:
- Outbound traffic goes via ISP-1
- Inbound traffic prefers ISP-1 (lower MED)
BGP Failover
Primary and Secondary ISP:
Scenario: Connection to primary ISP fails
Before:
BGP announces both ISP1 and ISP2 routes (ISP1 preferred)
Connection to ISP1 fails:
BGP detects: No heartbeat from ISP1
BGP withdraws: "Routes via ISP1 no longer available"
BGP announces: "Use ISP2 instead"
Result: Automatic failover within minutes
Key: BGP convergence time (usually 5-15 minutes) vs faster failover needed: use additional mechanisms (health checks, prefix hijacking detection)
BGP Terminology
Peering: Two routers exchanging routes:
Router A BGP Peering with Router B
├─ Establish TCP connection (port 179)
├─ Exchange BGP capabilities
├─ Send initial routing table
└─ Continuously send updates when changes occur
Convergence: Time for entire network to agree on routing:
Router A: "ISP connection down!"
All routers: Must learn new routes via Router B
Full convergence: 5-15 minutes typical
Flapping: Route keeps appearing/disappearing:
Route: 10.0.0.0/16 appears
↓ (30 seconds)
Route: Disappears
↓ (30 seconds)
Route: Reappears
Symptoms: Network instability, neighbor disconnections
Cause: Equipment malfunction, configuration error
BGP Configuration Best Practices
# Set timers appropriately
keepalive 3 # Send keepalive every 3 seconds
holdtime 9 # Expect response within 9 seconds
# Accept only expected routes
neighbor 203.0.113.1 filter-list
(only allow specific networks)
# Redistribute only necessary routes
redistribute connected # Include directly connected networks
# Set local preference for failover
route-map TO_ISP1 permit 10
set local-preference 200
neighbor 203.0.113.1 route-map TO_ISP1 inRoute Reflector (iBGP Optimization)
Problem: Full-mesh iBGP expensive
Network with 100 routers:
Full mesh = 100 × 99 / 2 = 4,950 iBGP sessions
Too many!
Solution: Route Reflector
┌─────────────┐
│ Route │ ← All other routers peer with this
│ Reflector │ (simpler topology)
└─────────────┘
│
├─ Router A
├─ Router B
├─ Router C
└─ Router D
Route Reflector:
- Receives routes from all routers
- Reflects them to other routers
- Like a hub for iBGP
BGP Security
RPKI (Resource Public Key Infrastructure) Verify route ownership:
BGP announcement: "I advertise 10.0.0.0/16"
RPKI check: "Does this AS own this IP?"
Unauthorized: Reject or warn
Prefix Hijacking Prevention
Attacker: "I advertise 8.8.8.8/32" (Google DNS)
With RPKI: Rejected (attacker doesn't own this IP)
Without RPKI: Might be accepted (big problem!)
Monitoring BGP
# View BGP neighbors
show ip bgp neighbors
# View BGP routing table
show ip bgp
# Show received routes for prefix
show ip bgp 10.0.0.0/8
# Show path to specific prefix
show ip bgp 8.8.8.8
# Output:
# BGP routing table entry for 8.8.8.8/32
# Paths: 3 available
# 64512 15169 (via ISP)
# 64513 15169 (via backup)
# (local as 65001)BGP Troubleshooting
BGP Session Down:
1. Check TCP port 179 connectivity
2. Verify BGP config (remote-as, neighbor IP)
3. Check AS numbers match expectations
4. Verify authentication (MD5) if enabled
5. Check timers not causing timeout
Route Not Learned:
1. Verify neighbor is advertising it
(show ip bgp neighbors … advertised-routes)
2. Check incoming filters
3. Verify AS path doesn't have loop detection
4. Check local preference/metrics
Route Flapping:
1. Check interface stability (ping continuously)
2. Check for configuration changes
3. Look for equipment errors
4. Enable damping to reduce flaps
Key Concepts
- BGP = Internet routing protocol (AS-level)
- AS (Autonomous System) = network under single authority
- eBGP = between different AS
- iBGP = within same AS
- Path attributes = decision factors (weight, local-pref, etc.)
- Convergence = 5-15 minutes typically
- Route reflector = simplifies iBGP topology
- RPKI = prevents route hijacking
- BGP uses policy-based routing, not just shortest path
- Understanding BGP critical for multi-site infrastructure