April 7-10, 2015 in San Diego, CA - PTT.br

Transcrição

Peering in Brazil - Americas Interconnection Summit - April 7-10, 2015 in San Diego, CA - PTT.br <[email protected]> - 1
Peering in Brazil
Americas Interconnection Summit
April 7-10, 2015 in San Diego, CA
Eduardo Ascenço Reis <[email protected]>
PTT.br Engineering Team <[email protected]>
Summary
This presentation will covers Brazil Internet and peering brief history,
PTTMetro/PTT.br peering, architecture and financial models,
some statistical data about the participants and traffic per locations.
Also critical points that create growth barriers and influence the future will be covered.
Brazil Facts in Summary
Source: The Brazilian Institute of Geography and Statistics – IBGE
27 Federative Units grouped in 5 Regions
São Paulo State
São Paulo State
Southeast Region
São Paulo City
São Paulo Population in 2015
City: > 12.000.000
Urban Area: > 20.000.000
CGI.br - The Brazilian Internet Steering Committee
CGI.br - The Brazilian Internet Steering Committee
The Brazilian Internet Steering Committee (CGI.br) was created by
Interministerial Ordinance 147, of May 31st, 1995, which was amended by
Presidential Decree 4,829 of September 3rd, 2003.
NIC.br - The Brazilian Network Information Center
CGI.br – NIC.br - CEPTRO.br
CGI.br
The Brazilian Internet Steering Committee (CGI.br) purpose is for coordinating and integrating
all Internet service initiatives in Brazil, as well as promoting technical quality, innovation
and the dissemination of the services available. The CGI.br is comprised of members from the
government, the corporate sector, the third sector and the academic community, and as such
constitutes a unique Internet governance model for the effective participation of society in decisions
involving network implementation, management and use.
NIC.br
The Brazilian Network Information Center (NIC.br) is a non-profit civil entity that implements
the decisions and projects designed by CGI.br.
CEPTRO.br
The Center of Study and Research in Network Technology and Operations (CEPTRO.br) is
responsible for services and projects that aim to improve the quality of Internet in Brazil,
working more specifically with Internet infrastructure.
PTTMetro/PTT.br is part of CEPTRO.br activities.
Reference Name
IXP - Internet eXchange Point
PTT – Ponto de Troca de Tráfego
Brazilian PTT History and Internet Key Points
Year
Location (City/State)
Event
1989
São Paulo / SP
.br TLD Activation
1991
São Paulo / SP
Brazil First Connection to Internet via FAPESP (Dawn of Brazil Internet)
1996
São Paulo / SP
PTT ANSP/FAPESP Operation Start
1997
São Paulo / SP
Registro.br Operation Start (Brazil NIR)
2000
Porto Alegre / RS
PTT RSIX Operation Start (RNP UFRGS)
2000
São Paulo / SP
PTT OPTiX-LA Operation Start (Optiglobe Inc, now Tivit)
2002
Curitiba / PR
PTT PriX Operation Start (RNP UFPR)
2002
Brasilia / DF
PTT FIX Operation Start (RNP)
2004
São Paulo / SP
PTTMetro (CGI.br)
2004
Rio de Janeiro / RJ
PTTMetro (CGI.br)
2004
Brasilia / DF
PTT FIX move to PTTMetro (CGI.br)
2004
Barueri / SP
PTT ANSP/FAPESP moved to NAP do Brasil (Terremark Latin America)
2005
Curitiba / PR
PTT PRIX moved to PTTMetro (CGI.br)
From Peering in Brazil presentation @ LACNIC XI / Salvador, BA – Brazil - Interconnection Schemes and Strategies for ISPs Tutorial
Brazilian PTT History and Internet Key Points
Year
Location (City/State)
Event
1989
São Paulo / SP
.br TLD Activation
1991
São Paulo / SP
Brazil First Connection to Internet via FAPESP (Dawn of Brazil Internet)
1996
São Paulo / SP
PTT ANSP/FAPESP Operation Start
1997
São Paulo / SP
Registro.br Operation Start (Brazil NIR)
2000
Porto Alegre / RS
PTT RSIX Operation Start (RNP UFRGS)
2000
São Paulo / SP
PTT OPTiX-LA Operation Start (Optiglobe Inc, now Tivit)
2002
Curitiba / PR
PTT PriX Operation Start (RNP UFPR)
2002
Brasilia / DF
PTT FIX Operation Start (RNP)
2004
São Paulo / SP
PTTMetro (CGI.br)
2004
Rio de Janeiro / RJ
PTTMetro (CGI.br)
2004
Brasilia / DF
PTT FIX move to PTTMetro (CGI.br)
2004
Barueri / SP
PTT ANSP/FAPESP moved to NAP do Brasil (Terremark Latin America)
2005
Curitiba / PR
PTT PRIX moved to PTTMetro (CGI.br)
From Peering in Brazil presentation @ LACNIC XI / Salvador, BA – Brazil - Interconnection Schemes and Strategies for ISPs Tutorial
Independent PTT Proliferation – Problem: Too Close Operation
From 1996 to 2004
São Paulo Metropolitan Area
PTT
Optix-LA
PTT
ANSP/FAPEP
PTT
Diveo
PTT
Telcomp
Customer
AS
(e.g. IDC)
Too many Connections
High Cost
From 1996 to 2004
PTT
Optix-LA
PTT
ANSP/FAPEP
PTT
Diveo
PTT
Telcomp
?
Customer
AS
(e.g. IDC)
From 1996 to 2004
PTT
Optix-LA
PTT
ANSP/FAPEP
PTT
Diveo
PTT
Telcomp
Customer
AS
(e.g. IDC)
IP Transit
ISP
NSP
PTTMetro – The Brazilian Metropolitan IXP Project
PTTMetro (PTT- Internet Exchange Point) is the project of the
Brazilian Internet Steering Committee (CGI.br) that provides the
necessary infrastructure for the direct interconnection between the
diverse networks that operate in a metropolitan region.
PTTMetro – PTT.br – Project # Operation
PTTMetro
CGI.br project started in 2004 and is still active.
PTT.br
NIC.br operation division responsible to deploy and maintain PTTMetro locations.
PTT.br has a partnership to support its operation with entities that host PIX
on all locations. Not complete list of PIX's hosts types:
● National Education and Research Network (RNP)
● Academic (e.g. Universities)
● Information Technology Government Companies (e.g. Procempa, Prodest, Prodepa)
● Internet Data Centers (IDC)
● Internet Service Providers (ISP)
● Internet Network Providers (NSP)
PTT.br – Actual Domain of Operation
BRAZIL
PTT.br 2004 – 3 Locations in Operation after PTTMetro Project was Started
●
●
●
Brasília
Rio de Janeiro
São Paulo
PTT.br 2015 - 25 Locations in Operation
Each location is an independent IX
NIC.br – Brazilian AS Distribution
NIC.br – Brazilian AS Distribution - Density Chart
http://ptt.br/localidades/brasmap
NIC.br – Brazilian AS Distribution
Brazilian Autonomous Systems (AS)
Total: 3221
Last Updated: 27/Mar/2015
NIC.br – Registro.br
National Internet Register (NIR)
NIC.br/PTT.br – IXP Locations and Brazilian AS Distribution
PTT.br – IXP Locations
All PTTMetro Locations are Isolated
The main purpose for PTTMetro locations is to allow the local traffic to
stay in the location.
The Brazilian Internet Steering Committee do not intend to interconnect PTTMetro
Locations and compete with telecommunications companies.
PTTMetro project stimulates and support inter locations transport services done
by participants companies.
PTTMetro have defined some transport modes in order to optimize common
resources and reduce costs that are already in production
(e.g. Campinas <=> São Paulo by Fasternet company)
PTT.br – Participants Distribution per Location
Sao Paulo
Curitiba
Florianopolis
Belo Horizonte
Vitoria
Belem
Sao Jose do Rio Preto
Cuiaba
Sao Carlos
Porto Alegre
Salvador
Londrina
Fortaleza
Maringa
Sao Jose dos Campos
Campina Grande
Lajeado
Rio de Janeiro
Campinas
Brasilia
Goiania
Recife
Natal
Manaus
Caxias do Sul
PTT.br – Some 2014 Highlights
- 608 Gbps / pico / day (141% growth/year)
- 26 localidades ( new Cuiabá - MT )
- 100 PIXes actives
- 788 participants ( Unique )
- 58 Switchs Installed ( nov13 to nov2014 )
- 13 DWDM Installed - Total 33 ( nov13 to nov2014 )
- 7 100GE Insterfaces Installed
- 1111 Total of 10GE interfaces (122% growth/year)
- 1812 Total of 1GE interfaces (6% growth/year)
CGI.br – NIC.br – PTT.br - Financial Model
Brazilians
(persons or companies)
R$
Domains / IP / ASN
Registration
.br
Return Benefits
Help Improve Internet
Authoritative Name Systems
DNS
.br
NIC.br divisions / activities
Registro.br
CERT.br
Cetic.br
Ceptro.br
PTT.br / IPv6.br/ SIMET.nic.br / etc
PTT.br - Financial Model HighLights
PTTMetro / PTT.br costs are sustained by CGI.br / NIC.br, despite
Autonomous Systems (AS) participants needs for ports (number and capacity).
Each participant AS has only costs to reach one or more Interconnection Points (PIX)
of a specific PTT.br location in order to access its peering fabric.
There is no startup and recurrent fees for AS participants at all PTT.br locations.
PTT.br – Operational Responsibilities
NIC.br
PTT.br is responsible for:
Installation, Management, Administration, Monitoring and Supporting all IX locations.
Always keeping Neutrality and High Quality as main targets.
Highlight: PTT.br is responsible to provide enough capacity to sustain bandwidth demand
on connection between PIX: (e.g. 1GE, 10GE, nx 10GE with DWDM).
PIX - Interconnection Point
Each entity that hosts a PTT.br PIX is responsible to provide continuous and high quality:
● IDC Infrastructure
Rack Space, Electrical Energy, Refrigeration, Physical Security,
Operational Hand, etc
● Dark Fiber Pair (preferably redundant) to Central PIX
PTT.br - Regional Metro Ethernet Non-Blocking Switch/Peering Fabric
AS A
Each PTT.br Location Infrastructure
Router
IXP
PIX A
AS D
Router
PIX D
PIX
Central
PIX B
Router
AS B
PIX C
Router
PIX: Interconnection Point
AS C
PTT.br – High Availability Network Topology Model
PIX Central
C1
PIX Central
C2
Core
Access
PIX
Remote
PIX: Interconnection Point
Dark Fiber Pair
PTT.br – São Paulo – PIX Topology
PTT.br – São Paulo Location – PIX Distribution
Central / Core PIX
C1 and C2
Remote / Access PIX
PTT.br – Peering Agreements
Multi-Lateral Peering Agreements (MLPA)
IP (v4/v6) traffic exchange between all participants.
BGP sessions established between participants and IXP route servers.
Different MLPA VLANs for IPv4 and IPv6.
Bilateral Peering Agreements (BPA)
Shared VLAN Mode
IP traffic exchange only between participants.
Use the same MLPA VLANs.
Dedicated VLAN Mode
IP traffic exchange only between participants (e.g. Internet Transit).
Use a dedicate VLAN for each peering agreement.
PTTMetro/PTT.br – Reference Model
Fundamental Definition
IP (v4|v6)
AS A
AS B
PTTMetro/PTT.br – Basic Operational Model – Allowed Traffic
AS A
MAC A
MAC B
IP (v4|v6)
PTT.br
Location
Peering Fabric
Allowed Frames with Ethertypes:
● 0x0800 - IPv4
● 0x0806 - ARP
● 0x86dd - IPv6
AS B
PTTMetro/PTT.br – Extended Operational Model – Inter AS Traffic
Allowed Multiple Connections to a Location for
Redundancy and/or Load Balancing
AS A
MAC A
MAC B
AS B
MAC C
AS C
PTT.br
Location
Peering Fabric
AS A
MAC A'
PTTMetro/PTT.br – Extended Operational Model – Intra AS Traffic
Denied Multiple Connections to a Location for Internal Traffic (local loop)
AS A
MAC A
MAC B
AS B
MAC C
AS C
PTT.br
Location
Peering Fabric
AS A
MAC A'
PTTMetro/PTT.br – Extended Operational Model – Multiple AS per Single Port
Shared L2 structure outside PTT.br operation domain
AS A
MAC A
NSP
Transport
Provider
AS D
MAC D
PTT.br
Location
Peering Fabric
MLPA
PTTMetro/PTT.br – Extended Operational Model – Multiple AS per Single Port
Use Different VLAN Tags for Logical Isolation (IEEE 802.1Q)
AS A
MAC A
VLAN
A
VLAN
A
NSP
Transport
Provider
VLAN
B
AS D
MAC D
VLAN
B
PTT.br
Location
Peering Fabric
MLPA
VLAN C
PTT.br – All Locations Aggregated Traffic
Monday, 30th March 2015
PTT.br – Stacked Locations Traffic
Monday, 30th March 2015
PTT.br – All Locations Aggregated Monthly Exchanged Traffic Growth
160
140
120
Petabytes
100
80
60
40
20
0
March 2015: almost 150 PetaBytes
PTT.br – Representation to Brazil Internet Traffic
Common Question:
But how much Brazilian Internet traffic is exchanged through PTT.br?
Not simple answer.
Missing Data:
● Not all Brazilian AS are connected to PTT.br IX.
● Most AS do not share their values of Internet traffic.
PTT.br – Internet Routing Relevance Analyses
$ telnet lg.sp.ptt.br
lg.sp.ptt.br> show ip bgp summary
BGP router identifier 187.16.218.252, local AS number 20121
RIB entries 39978, using 3748 KiB of memory
Peers 8, using 36 KiB of memory
Neighbor
187.16.216.253
187.16.216.254
187.16.223.253
187.16.223.254
V
4
4
4
4
AS
26162
26162
26162
26162
MsgRcvd MsgSent
1670067 705590
2009823 705372
1671526 705594
2096128 705578
TblVer
0
0
0
0
InQ OutQ Up/Down State/PfxRcd
0
0 02w3d08h
23577
0
0 01w6d06h
23669
0
0 02w3d08h
23035
0
0 02w3d08h
23045
Total number of neighbors 4
lg.sp.ptt.br>
ASN 26162 – PTT.br MLPA Route Servers
Output taken in 20150404 – 01am (UTC-3)
IPv4 Address Space Allocated to Brazilian Entities
ftp://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-latest
Grep lines for “|BR|ipv4|” occurrence and extract netblocks
São Paulo (SPO) PTT.br
Multi-Lateral Peering Agreements (MLPA)
BGP Routing Table
lg.sp.ptt.br.rib.20150321.1947
Process and extract IPv4 SPO PTT.br Brazilian Prefixes
IPv4 Address Space Allocated to Brazilian Entities
ftp://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-latest
Grep lines for “|BR|ipv4|” occurrence and extract netblocks
Global Internet Routing Table Sample
University of Oregon Route Views Project
http://archive.routeviews.org/bgpdata/2015.03/RIBS/rib.20150322.0200.bz2
Process and extract IPv4 Global Brazilian Prefixes
IPv4 routed address space analyses based on /24 prefixes
Oregon
PTT.br
Global Internet
SPO MLPA
307459
83222
Unique
Common
Unique
224545
82914
308
72,96%
26,94%
0,10%
TOTAL
390681
307767
IPv4 routed address space analyses based on /24 prefixes
Oregon
PTT.br
Global Internet
SPO MLPA
307459
83222
Unique
Common
Unique
224545
82914
308
72,96%
26,94%
0,10%
TOTAL
390681
307767
Almost 27% of Brazilian IPv4 routed address space is present at São Paulo PTT.br IX
PTT.br – São Paulo – MLPA Participants Diversity
TITAN
NETRONIX
JETCELL
GIGA
COMDA
3COM
UNK1
SERVERU
ASUSTEK
NIC.BR
IBM
LANNER
HEWLETT
INTEL
ROUTERBOARD.COM
0
50
100
150
200
250
ROUTERBOARD.COM
CISCO
DELL
HEWLETT
EXTREME
JUNIPER
INTEL
BROADCOM
BROCADE
LANNER
LANNER
EXTREME
BROCADE
HEWLETT
BROADCOM
DELL
INTEL
CISCO
JUNIPER
Router
Switch
ROUTERBOARD.COM
0
50
100
150
200
250
MLPA: 625 Active Routers
RouterBoard
Server
PTT.br – Barriers for Brazil Peering Growth
1. Critical Infrastructure
Resources Availability and Quality
Data Centers
● Electric Power Feed
● Cooling Systems
External Optical Fibers
PTT.br – Barriers for Brazil Peering Growth
2. Autonomous Systems (AS) Culture
Result in:
Small number of Brazilians AS compared to its potential
(number of ISP, medium and big companies, Universities and others)
A significative number of entities that received ASN and netblocks (IPv4 and IPv6)
are not technically prepared to operate the AS
PTT.br – PTT-RJ – Olympic and Paralympic Games - Rio 2016
Olympics Games - 5th to 21th August
Paralympic Games 7th to 18th September
Less then 500 days to start the Games
PTT.br Operating freezing on
Rio de Janeiro and Sao Paulo Locations
- From July 25th 2016 to 18th September 2016
- Start 10 days before the games
- Ends just after the Paralympic Games
http://www.rio2016.com.br
PTT.br – IX Associations Member
European Internet Exchange Association (Euro-IX)
https://www.euro-ix.net/
PTT.br is an Euro-IX member since 2010
with personal attendance on meetings since 2011.
LAC-IX
Latino American and Caribean
Internet Exchange Asociation (LAC-IX)
http://lac-ix.org/
PTT.br is a cofounder and member of LAC-IX.
PTT.br – Internet Governance Forum - IGF 2015
IGF 2015
The Tenth Annual IGF Meeting is scheduled to take place in João Pessoa, Brazil,
on 10-13 November 2015.
The 2015 edition will hold 3 workshop proposals dealing with the IXPs issues.
There will be one Best Practice forum on IXPs, which intend to be a space of collaboration
of multiple actors involved with Internet traffic all over the world.
The objective is to present and discuss the best practices of various IXP projects in global and
local levels, so as to promote collective learning and to propose new strategies and standards for the sector.
http://www.intgovforum.org/cms/aboutigf
http://www.intgovforum.org/cms/best-practice-forums
http://www.intgovforum.org/cms/workshop-proposals/list-of-published-workshop-proposals
PTT.br – PTT Forum 9 – São Paulo, Brazil
PTT Forum 9
http://ptt.br/pttforum/
The PTT Fórum 9 event will take place in São Paulo, Brazil on the 7th and 8th of
December 2015.
PTT Fórum normal audience is composed by Brazilian Autonomous Systems (AS) and
PTTMetro/PTT.br IX participants (some of them are international companies).
The event will be translated between Portuguese and English.
The event will be part of the Brazilian Internet Infrastructure week together with an
IPv6 Forum and GTER/GTS (Network Engineering, Operation and Security Working
Groups).
PTT.br – Thanks & Contact
Thanks | Obrigado
http://ptt.br/
[email protected]

April 7-10, 2015 in San Diego, CA - PTT.br

Transcrição

Documentos relacionados

PTTMetro – PTT.br

Lógica de Programação