BGP experimentation for fun and not profit.

Most of the internet is powered in some way by BGP, the Border Gateway Protocol. BGP is used to exchange routes between systems on the internet. It's one of the fundamental protocols used to allow different networks to communicate with each other.

BGP itself is super easy to play with in a lab. Set up a few UNIX VMs running BIRD and a virtual switch and your ready to go. This is cool, and is similar to how BGP works in the wild at an IXP. If you want to go one step further, check out dn42. dn42 is a dynamic VPN powered by BGP. It gives you the experience of creating objects in the registry and connecting to others via a tunnel. You also get access to some real services that people host on dn42. Both a private lab and dn42 are lots of fun and provide great learning opportunities, but I wanted to try out BGP on the real internet. Some may argue this is a horrible idea, and I do want to give a warning here; The internet is fragile, and experimenting with BGP can really mess things up. In this post I'm going to go over how to experiment with BGP in the wild without creating issues for other people. I HIGHLY recommend you try out a private lab or dn42 before messing with the real internet.

Goals

My goal of this project was to announce my own IP block and obtain network connectivity to/from a single server, learning about BGP and the internet along the way. Spoiler, I did achieve this goal. Keep in mind the single server part. In this post I'm not going to go over how multicast/anycast works.

Disclaimer

Before we start, it's important to know that I'm not a professional. I'm a student interested in IT, with no formal training. Its highly likely that I will mess up, say something wrong, lead you in the wrong direction, etc. I'm not responsible if you follow this guide and mess anything up. By continuing to read, you agree to not hold me liable. Please send me an email with any questions, comments, corrections, or anything else you want to say. I do my best to reply to all emails that I receive. With that, lets get started.

Terminology

What you'll need

The first step is actually figuring out what all you need to make it work. What you'll need for this project is:

  1. An ASN (Autonomous System Number)
  2. IP block (Could be IPv4 or IPv6)
  3. Upstream providers (Bigger networks that will announce your block to others)
  4. A server with internet connectivity (This could be at home or elsewhere)
  5. A little bit of money. (I'll go over this later, but you will likely need to pay some for the IPs and ASN)

Choosing your providers

The first thing you need to figure out is what your upstream providers will be. You don't really need an actual contract, since most will want to know your ASN (Which by this point you won't have.) I picked Hurricane Electric, since they offer a 100% free BGP tunnel for IPv6. Keep in mind this is IPv6 only. I decided that I was going to start with only IPv6 overall, since IPv4 address space is way harder to get now. I also picked Vultr (technically Choopa). I had initially wanted to host a VPS with Vultr, and have all the IPs go there. I later didn't want to pay a monthly fee, so I actually never ended up peering with Choopa. There's also Netassist, a company based in Ukraine that offers free BGP IPv6 tunnels as well. Its really up to you, but for now pick 2. You can always change later without too much trouble. I picked Vultr and Hurricane Electric.

Registering an ASN

I knew right off the bat that I wanted my own ASN. Its possible to announce an IP block with your providers ASN. For example if you have a business ISP, you might be able to get them to announce your block for you. We aren't going to go over this, since I wanted to do it all from scratch. (At least as much as possible.) The first thing I did was go looking for an ASN. The first logical choice was to go straight to my RIR. The image below shows each LIR and what region they cover.

RIR Map

It's important to note that as long as your network has presence in the specific country, most RIRs will allow you to be physically located in another region. I'll start with ARIN. ARIN covers North America, and is super expensive. Customers go straight to ARIN, not through a LIR. This means that you have to pay a monthly and one time fee, depending on what resources (IPv4, IPv6, ASN) that you would like. This was out of the question, as this can rack up to ~$500/month. Way out of my price range. The next logical step was going to LowEndTalk and checking out any offers there. I found a very helpful all in one IP thread. There, I found that RIPE and APNIC LIRs had the most offers. With RIPE being the cheapest, I sent off a few emails and a few discord messages on the LET discord server to LIRs in the RIPE region. I got in contact with a few very nice people, and started the process of applying for an ASN. Only one little problem; RIPE doesn't like minors! For me, this was the end of RIPE. If your over the age of 18, and have a valid government ID (passport/drivers license) then I RIPE is probably the best and cheapest option. A very nice LIR referred me to a friend that's a LIR with APNIC. This particular LIR offers an ASN, and a /48 IPv6 block for $15/year. Many RIPE LIRs charge a 1 time fee, so this want ideal. But after talking with the LIR a bit and figuring out all my options, I was happy to pay. $15/year was totally doable for me when considering the other options (I didn't have any.) Its also possible to transfer your ASN to another LIR, so if I wanted to change in the future, its not a problem. I asked about ID and being under 18 and he said with APNIC they don't even ask for ID. Score! So I gave him the necessary information. 2 different upstream provider ASNs, my name, organization (can be your name), phone number, and email. 3 days later, I got a message back with my ASN! I payed him $15 via paypal and I was set. Now I had my ASN and IP block. Hurricane Electric provides a great site to look up BGP information. Head to https://bgp.he.net/ and input your ASN. Check the WHOIS and IRR tabs to make sure your name/organization shows up. If not, contact your LIR.

Announcing your block

Now I was ready to move on to actually setting up the BGP session. For this, you will need a server. I had a VPS at DigitalOcean, so I used that. Check out lowendtalk.com for lots of great deals. When looking for a VPS, make sure it has a dedicated IPv4 address. You don't really need IPv6 for this, but if its free then its not bad to get as well. For the tunnel, I started with Hurricane Electric since they have tunnel servers all around the world. I went to (https://tunnelbroker.net), made an account, and used this form to request a BGP tunnel. For IPv4 endpoint, put the IPv4 address of the VPS/server tha tyou have. Enter your ASN, and the IPv6 prefix that you got form your LIR. (Something like 2001:DB8::/48) I selected the tunnel server thats geographically closest to my server, and clicked create. A day later, I got an email from them requesting a LoA (Letter of Authorization/Agency). This a letter from the IP block owner (Your LIR) authorizing you to announce the block. I contacted my LIR and was instructed to write my own LoA, and then check in with him to make sure it's all good. Below is a template of a LoA. Fill it out with your information, and forward it to your LIR to make sure they approve.

TODAY's DATE

LETTER OF AUTHORIZATION

To Whom It May Concern:

YOUR NAME authorizes PROVIDER NAME (PROVIDER'S ASN) to announce the following route blocks.
This agency shall remain in effect until revoked or modified by YOUR NAME in writing.

2001:DB8::/48

By signing below, I certify that I am authorized on behalf of YOUR LIR to execute this Letter Of Agency.

Sincerely,

YOUR NAME

I then replied to Hurricane Electric's email with my LoA attached. They verify against WHOIS and IRR listings, so contact your LIR to make sure. An hour later they replied telling me that their side was all set, and to peer with ::1 of the tunnel's /64 allocation. Ok, there's a lot in that sentence. What that means, is that you need to set up a 6in4 tunnel, and peer with the first address in that block.

Setting up your router

I decided to use a Debian 10 VPS. I'm assuming that you have a fresh install. The first thing to do is configure your IPv6 tunnel to HE.net Assuming the following information open up /etc/network/interfaces and add this block, substituting your IPs for your HE.net tunnel information in this example.

Key:
Your server's public IPv4 address: 172.16.34.125
HE.net tunnel server IPv4 address: 192.168.221.152
HE.net peer address: 2001:DB8:FC:CE::/64
Your IPv6 prefix: 2001:DB8:EC::/48
Your ASN: 012345
auto he-ipv6
iface he-ipv6 inet6 v4tunnel
        address 2001:DB8:FC:CE::2
        netmask 64
        endpoint 192.168.221.152
        local 172.16.34.125
        ttl 255
        gateway 2001:DB8:FC:CE::1

Then run the following commands to apply the changes, then reboot.

sudo modprobe ipv6
sudo ip link set dev he-ipv6 down
sudo ip link set dev he-ipv6 up

There are a few routing daemons to use with linux. I chose BIRD. Install it with apt like so:

sudo apt install bird

Bird is actually 2 parts, bird and bird6. We're only using bird6 for this project, since its an IPv6 only network.

Edit /etc/bird/bird.conf

router id 172.16.34.125;

protocol kernel {
    scan time 60;
    import none;
}

protocol device {
    scan time 60;
}

Open up /etc/bird/bird6.conf in your favorite text editor.

listen bgp v6only;

router id 172.16.34.125;

protocol device {
        scan time 10;
}

protocol kernel {
        export all;
        scan time 15;
}

protocol static static_bgp {
        route 2403:cfc0:1015::/48 reject;
}

protocol bgp {
        export where proto = "static_bgp";

        local as 012345;
        neighbor 2001:DB8:FC:CE::2 as 6939;
        source address 2001:DB8:FC:CE::2;
}

Remember to subsitute ALL the IP addresses in the config templates above with your respective tunnel IPs. (Its all in key)

Once BIRD is configured, restart it. sudo systemctl restart bird If everything up until this point went right, running birdc6 show protocols all bgp1 should show BGP state: Established. It may take a minute to establish the BGP session and exchange all the routes, so you might have to rerun that command to check your session status.

Configuring your new interface

Now that you have a session established, you need to set up your server as a packet-forwarding device (router). First, set up IPv6 forwarding by editing /etc/sysctl.conf and adding net.ipv6.conf.all.forwarding=1 to the end of the file.

Next, set up the dummy interface. This will be the interface that has your new subnet.

sudo ip link add dev dummy1 type dummy
sudo ip addr add dev dummy1 2001:DB8:EC::/48
sudo ip -6 route add :: dev dummy1

Then reboot, and make sure all the interface is there and the services are started. Run birdc6 show protocols all bgp1 to make sure your session is still established, and if so, test IPv6 connectivity!

curl -6 https://ifconfig.co

If you get back your IPv6 prefix, you've done it! You can now use your IPv6 subnet to host whatever you like!

I've had a ton of fun with this project. If you have any questions, comments, concerns, or just want to say hi, send me an email: [email protected] I really enjoy hearing your feedback.