System- and Network- wide bandwidth shaping for P2P apps

At my home, with 5 fixed IP's on Pacbell DSL, I have 1.5Mbit downstream but a meager 128k upstream. Recently, I installed Morpheus on my Win98 box and, predictably, with my fixed IP and always on machine, I guess I became a popular and high-reliability supernode.

I also began, predictably, to notice that surfing the web on this machine could be slow and painful. That shouldn't be a problem- I have 1.5Mbit down- but the 128k up was so full that the small HTTP GET requests that my browser sends had to wait in line to get out. So it was really a problem of latency, not bandwidth, and anyone with wireless 128k from the dying Metricom will tell you, they'd sometimes rather surf the web with (relatively) low latency modem than a high latency wireless.

Single-Machine Bandwidth Balancing

Now, this problem could be solved with software. All that would be necessary is if my P2P app could tell that I was clicking a link in my browser client, and it could hold off, for just a moment, on sending any data until the tiny request could go out, get established, and then the p2p upload could resume.

From the perspective of the P2P app, we are limiting it's bandwidth. From the perspective of the Web browser, we are guaranteeing that it will have a minimum amount of bandwidth and delay. So throttling or shaping can be used to implement "QOS" or Quality of Service.

Servers, routers, gateways, cable modems, server apps like FTP, Apache, and even OS's like linux have throttling, allowing you to set exact limits for how much bandwidth should go through. This is how web hosters make sure one website doesn't hog the machine from all the other hosted sites, or how Cable modem operators (attempt) to keep the bandwidth fairly spread to the users.

Back to me: I have a Linux box on this LAN said, "Why was my site so slow the other day?" The reason was, of course, that the Win98 machine was using all the bandwidth uploading the Phantom Edit and/or 40's jazz music to others, and my wife's attempt to view her own page was inhibited by this.

In this case, the problem was indeed that there was too much upstream traffic, which directly blocked the outgoing traffic of the webpage. But another factor existed as well: even if there had been some system-level software on the Win98 box to detect web browsers on that machine, Morpheus couldn't possibly know that my linux machine on the same network was trying to serve!

Home Network Bandwidth Balancing

This could be solved in two other ways: One, if there was some sort of two-way protocol between Morpheus and other bandwidth hungry apps, such that apache could send across the network (quickly enough):

Apache: "Hey, Morpheus, I need to send a web page, hold off for a second"

Morpheus: "No prob."

Apache: "Ok I'm done".

Of course this conversation would be very rapid and happen many times per second.

Another solution (more likely) is that the techniques migrate down to the gateway or DSL router and some application toolkit goes to the OS level. Then, some local reservation of service would go like this:

Apache: "Hey, gateway, I'm on this linux machine and I'm a service app. I am HTTP, tcp, short connection oriented- I need pretty low latency because there's a lot of disconnect/connect. I'm the most important app on this network. I don't want to drown out other apps however."

Gateway: "Ok lemme see... ok, Apache."

Morpheus: "Hey, gateway, I run all the time. I have fairly long connection periods compared to connects... but I'm lower priority. Don't let me mess up the web services."

Gateway: "Ok gotcha."

Now, if the website is a bandwidth intensive site and the P2P app is very web-serverish with many small transactions, these profiles are not always correct... the point is, it would be nice if the various apps settled into a rhythm and worked together, if not perfectly, in the optimal way.

---

I always compare the bandwidth problem to the world hunger problem: Given the proper distribution, there is enough grain to feed everybody. The problem is getting it there. Similarly, there is always unused bandwidth somewhere on the Internet. Even on your home LAN, there is often enough bandwidth, if you distribute it right.

It took years for OS's to multitask smoothly and balance interactive (user) response with computation and disk use. We now need to see it happen for network applications, and get that multi-tasking effectiveness for our bandwidth.

Some potential solutions (and these have all been invented hundreds of times, they just haven't trickled down to the end user machine) are:

• System-wide bandwidth reservation, a software API for accessing it, allowing one machine to make the best use of it's bandwidth.
• Centralized (on a small lan) reservation of bandwidth, perhaps centered around a gateway device or DSL/Cable access point
• Peer-to-peer cooperative reservation of bandwidth, where apps sort of work it out amongst each other (on a network) how they will share the bandwidth