net p2p: writing peer-to-peer networked apps with the microsoft .net framework
lance olson
this article assumes you’re familiar with http and .net
level of difficulty 1 2 3
download the code for this article: netpeers.exe(66kb)
browse the code for this article at code center: sharebaby
summary peer-to-peer applications such as napster, gnutella, and scour that communicate as peers sharing and receiving information are becoming commonplace as a means for users connected on large networks to take advantage of the vast resources available to them. the microsoft .net framework provides a rich platform for building p2p apps.
this article explains the concepts that make up peer-to-peer applications. the peer-to-peer application model, discovering other peers, and querying peers for information are discussed. the article goes on to cover the system.net namespace for the use of internet protocols, the system.web.services namespace for exposing web services, and firewall and port issues. finally, the role of the .net framework in simplifying the design of powerful peer-to-peer applications is outlined.
--------------------------------------------------------------------------------
he microsoft® .net framework sdk is a common framework of classes designed to provide a rich platform for building applications. in this article, i'll explain how this framework can be used to create peer-to-peer (p2p) applications and discuss the design decisions that are important to consider when writing a peer-to-peer application. i'll look at the .net framework support for the particular features listed in figure 1, and discuss how they can help you.
what is a peer-to-peer application?
communication is a key element when writing nearly any type of application. an application gains value when it becomes distributed and interacts with other resources available to it on the internet or intranet. the most common model for communication over the internet today is client/server, where there is a client that knows how to request information and post information to a server, and the server knows how to respond to requests from the client. a browser talking to a web server is a common example of this model. many browsers can send requests to the web server, and the server does its job by listening for those requests and responding back to each of the browsers requesting or sending information (usually in the form of web pages). in this model, the web server cannot arbitrarily contact the browser. the "conversation" is always initiated by the client.
a peer-to-peer application is different from the traditional client/server model because the applications involved act as both clients and servers. that is to say, while they are able to request information from other servers, they also have the ability to act as a server and respond to requests for information from other clients at the same time. this approach increases the amount of value that each node on the network can add because it not only takes information from a source, but it also has the ability to share that information with other sources. a typical peer-to-peer application has the following key features that help define it:
discovering other peers the application must be able to find other applications that are willing to share information. historically, the application finds these peers by registering with a central server that maintains a list of all applications currently willing to share and giving that list to any new applications as they connect to the network. however, there are other means available, such as network broadcasting or discovery algorithms.
querying peers for content once these peers have been discovered, the application can ask them for the content that is desired by the application. content requests often come from users, but it is highly conceivable that the peer-to-peer application is running on its own and performing its query as a result of some other network request that came to it rather than a specific request made by a user at that machine.
sharing content with other peers in the same way that the peer can ask others for content, it can also share content after it has been discovered.
there are a number of design options to consider when designing a peer-to-peer application using the .net framework. the decisions you make about the architecture for your peer-to-peer application will have a significant impact on the type of features that your application is able to offer and, therefore, the experience that your users will have when using your app. the range of applications in this area can be thought of as a continuum from what i'll call pure peer-to-peer to client/server. in the next sections, i'll take a look at a few of the choices.
pure peer-to-peer
a pure peer-to-peer application has no central server whatsoever, as you can see in figure 2. it dynamically discovers other peers on the network and interacts with each of them for sending and receiving content. the strength of this type of application is that it does not rely on any one server to be available for registration of its location in order for other peers to find it. at the same time, the lack of a central discovery server poses a problem because a relatively low number of clients can be discovered, thereby limiting the application's reach. in this scenario, a peer can either use information from a local configuration scheme to discover the clients (for example, a configuration entry that tells it who to talk with) or it can employ network broadcasting and discovery techniques such as ip multicast to discover the other peers.
figure 2 a pure p2p
using ip multicast can be problematic since it is not widely deployed on the internet, but it can be useful in intranet scenarios where the network is more controlled and infrastructure required for multicast is known to exist. pure peer-to-peer is also being deployed on the internet in cases where non-multicast schemes are used to discover the peers. in this case, the applications use some other scheme such as a well-known node approach, where each peer knows about at least one other peer and they share this knowledge with other peers to form a loosely connected mass of nodes.
peer-to-peer with a simple discovery server
this architecture, shown in figure 3, works just like the pure peer-to-peer architecture except that it relies on a central server for discovery of the other peers. in this model, the application usually notifies the central server of its existence at startup time (or login time). the peer application then uses this server to download a list of the other peers participating on its network that it can use to query for content. when content is needed, it goes through the list and contacts each peer individually with its request.
figure 3 p2p with a simple discovery server
in many cases, it is easier to make this solution scale better than the pure peer-to-peer option because it circumvents the issues of discovery by requiring only one request to the central server. note that it is possible to make pure peer-to-peer solutions scale extremely well, but if you are able to rely on a server for some of the basic tasks (like discovery), high scalability can be achieved with a lower cost in terms of development time. however, this approach hinges on the availability of the central server. if the central server is not available, the peer-to-peer application will not be able to find other peers.
in addition, requesting content from each individual peer can be quite expensive from a network resource perspective. this may not seem like a big deal if you're thinking about a few peers interacting over a network, but if your app is being written for use over the internet or a large enterprise environment, this consideration suddenly becomes much more significant, scaling factorially.
peer-to-peer with a discovery and lookup server
this model, similar to the one shown in figure 3, extends the discovery server so that it also includes content lookup services. in this case, the peer application not only registers with a discovery server, but it also uploads a list of its contents at regular intervals. when an application is looking for some particular content, it queries the central server rather than sending a query to each client. the central server then responds with a list of the clients that contain the requested content, and the peer application can contact those clients directly to retrieve the content.
quite often this approach will scale better than the previous options because it reduces the number of queries going over the network (arguably one of the scarcest resources). however, this saving will incur a cost on the server. servers are now more involved in the process of content sharing and the peer's demands will use significant resources.
p2p with a discovery, lookup, and content server
just to show that this can actually come full circle, a system can be designed so that the peers can upload the content to the server as well, if you so choose (see figure 4). this approach effectively becomes the client/server model because the peers are no longer contacting other peers for content. each peer registers with a server (if needed), queries it for information, and transfers any desired content down from the server. the problem with this approach is that when content is downloaded from all of the clients, the server quickly becomes the bottleneck and is easily overwhelmed by the peers (clients).
figure 4 p2p with a discovery, lookup, and content server
to put this into perspective, consider a peer-to-peer application that shares video content. let's assume the application supports up to 100,000 peers, each containing megabytes of data. the total amount of content available to any one peer can quickly reach into hundreds of terabytes. while server capacity can always grow, placing such demand on the server can be costly and can create a significant number of bottleneck and reliability issues on the network. placing all of the demand on the server also means that the substantial resources available on the clients that would be utilized in the peer-to-peer model are potentially wasted in the client/server model.
application model for peer-to-peer
the .net framework has a significant range of choices when it comes to the type of application that you can create. when writing your peer-to-peer application, it is important to understand how it will be used. this will make it easier to decide which .net application model to use. in the case of peer-to-peer, four powerful application models (or application types) are available. a brief overview of these models follows.
web services found in the system.web.services namespace, the web services technology provides an excellent way to handle registration, discovery, and content lookup for your peer-to-peer application. a web service allows you to quickly write a class that listens for incoming requests, processes them as they arrive, and sends back useful information in the form of objects that are easily understood by the peer application. an example later in this article shows how a web service for a peer-to-peer application might be implemented.
windows forms found in the system.winforms namespace, windows® forms is the .net framework solution for writing the type of rich windows-based gui applications that help to make the peer-to-peer experience much more exciting. windows forms is the ideal technology for writing the gui for the peer that lets your users log in, request, and share content.
web forms found in the system.web namespace, the web forms technology makes it very easy to return html content to a peer application. this can be useful if you want to spice up your peer-to-peer application with general content about the service or advertisements about using the service. when the peer-to-peer application starts up and registers with the web service, it can also call a web forms application to get the latest html content from the server.
service process found in the system.serviceprocess namespace, a service process (also known as a windows nt® service) is useful in peer-to-peer scenarios as a long-lived discovery server. in most cases, a web service is better for fulfilling the role of the discovery service (mentioned previously). however, in cases where the discovery mechanism is not using the http protocol, a service process listening for some other protocol might be the best way to go.
in addition to providing rich choices for application models, the .net framework drastically simplifies the networking side of peer-to-peer application creation, thanks to the classes found in the system.net namespace and the system.web.services namespace. let's take a closer look at each of these namespaces, the classes they provide, and applications built around each.
system.web.services namespace
the system.web.service namespace contains classes for consuming and exposing a web service. a web service in the .net framework is programmable application logic that is accessible via the simple object access protocol (soap). soap is a w3c-submitted note that uses standards-based technologies (http as a transport and xml for data description) to encode and transmit application data. consumers of a web service do not need to know anything about the platform, object model, or programming language used to implement the service; they only need to understand how to send and receive soap messages (which are simply composed of xml sent over http).
with the .net framework, creating a web service is as simple as creating a class in a page on the server. likewise, consuming that web service from the peer application is extremely easy and is accomplished by calling a method on a proxy class that is generated with visual studio.net or with the webserviceutil.exe program in the .net framework sdk. for more general details on how web services work, check out the quickstart documentation in the sdk.
figure 5 shows the code you might write in a web service to expose an api on the server that is called by the peer application for registration and file lookup. note that in this case i am mixing client/server technology with peer-to-peer concepts to create an application that is fairly simple due to the traditional nature of client/server, yet extremely powerful because it is able to use the resources of all peers registered on the network.
first, you'll notice a class called p2pservice. this class contains the methods that you'll be calling to interact with the service. in addition to a constructor, you'll see the class's methods described in figure 6.
you'll also notice a simple peerfile class. this class is used as a container to simplify the storage of peer file information. it contains the following members:
ipaddress, which represents the peer's ip network address
name, which contains the name of the file
connection, which contains the connection type and speed through which the peer is connecting
size, which denotes the size of the file
for the sake of simplicity, the implementation of these methods is mostly blank in this article; however, the sharebaby sample that i've provided (downloadable from the link at the top of this article) contains a full working implementation.
next, the corresponding client code in figure 7 shows the proxy class that has been built against the web service using the webserviceutil.exe tool or using a web reference in visual studio.net.
system.net namespace
the system.net namespace contains classes that provide support for creating applications that use internet protocols to send and receive data. using a layered approach, the net classes enable applications to access the network with varying levels of control, depending upon the needs of the application. the spectrum covered by these levels includes nearly every scenario on the internet today—from fine-grained control over sockets to a generic request/response model. furthermore, the model is extensible so that it will continue to work with your application as the internet evolves. for more general details on how the net classes work, check out the networking section of the quickstart documentation in the sdk.
in figure 8 and figure 9, you'll see how the net classes can be used to transfer a file from one peer to another. this code is divided into two methods. the first method, shown in figure 8, is called listenforpeers. this method puts the peer application in a listening mode so that it can respond when any other peers try to communicate with it. once another peer does contact it, the listening peer will proceed to read in the name of the file being requested and then write back the data that represents the requested file. running this code in the main part of an application form would cause the application to appear as though it had hung while waiting for a request to arrive because this method will block processing while it waits for a peer to contact it. in sharebaby, this method is called on a thread that is distinct from the main peer, so the user application remains responsive even when no requests for content are being processed.
the second method, shown in figure 9, is called downloadtoclient. this method will contact a peer (who is listening for requests) and issue a request that it transfer a particular file. if the listening peer has the file, the content is then transferred to the peer application making the request with downloadtoclient.
in addition to being a useful tool for sharing distributed information, a peer application can also be integrated with traditional web browsing technologies to add html content to the user experience. doing this turns the application into somewhat of a hybrid between a browser and a peer application. for example, there are many cases where your peer-to-peer application can provide opportunities for including up-to-date visual information about your peer-to-peer service or as a tool for advertising that supports your service. you could also build a peer-to-peer app for sharing corporate information. in it, documents and presentations can be shared using the peer service, while general company information such as benefits and hr information can be displayed in a traditional browser window. the code in figure 10 shows how the webrequest and webresponse classes in system.net can be used to download html content from a web server.
firewalls and ports
the peer application in my example is listening for a connection on port 8081. the selection of ports and the protocol used by the peers should be taken into account when designing the application because firewalls are often designed to allow communication only over a particular port. in the case of sharebaby, port 8081 may be fine for sending and receiving content in a particular network or directly on the internet, but it probably won't work through a corporate firewall because that port is likely to be closed. if you are designing your peer-to-peer application for a corporate environment, it might be useful to talk with your network administrator about which ports are open or recommended for application use. if the application is designed for users who will most likely not be going through a firewall to find other peers, just about any port that isn't already reserved for another protocol will work. in general, port numbers are divided into three ranges:
well-known ports (from 0 through 1023)
registered ports (from 1024 through 49151)
dynamic and/or private ports (from 49152 through 65535)
for a list of the well-known and registered ports, check out the iana port assignments list at http://www.isi.edu/in-notes/iana/assignments/port-numbers. in most cases, applications intended for general use over the internet should use a port in the dynamic or private range or register.
scalability
several of the options mentioned in this article can affect the scalability of your peer application. network bandwidth is often the scarcest resource (hence the bottleneck) in a peer-to-peer application. careful consideration should be given to the trade-offs between the flexibility of the peer application to discover other peers and query them for content, and the application's ability to scale up to hundreds of thousands of clients. unfortunately, further discussion of these issues is beyond the scope of this article.
a sample peer-to-peer application
the sharebaby application is a working example of a peer-to-peer with a discovery/lookup server. it illustrates the use of the system.web.service namespace to expose a sharebaby web service that is then consumed by sharebaby peer applications. these peers use the sharebaby service to register the names of the files they want to share and to query for the locations of files they are interested in obtaining from another peer. to represent the sharebaby peer application, the sharebaby sample demonstrates the use of the windows forms application model discussed earlier in this article. finally, sharebaby uses the system.net namespace in order to demonstrate how content is transferred from one peer to another.
conclusion
the peer-to-peer application model applied to large networks such as the internet offers an exciting opportunity to share information and content over a network on a scale that has never been seen before. the .net framework and visual studio.net make it easier for developers to write peer applications by offering a rich ui client programming model, support for scalable web services, and easy-to-use networking classes, all within the same powerful development environment.
