figure 3 caching
the amount of data you can reach at any level is different, but the right doses are determined on a per-application basis.
also different from layer to layer is the time needed to retrieve data. session, in most cases, is an in-process and in-memory object. nothing could be faster. keeping session lean is critical because it is duplicated for each connected user. for quick access to data that can be shared between users, nothing is better than cache or application. cache is faster and provides for automatic decay and prioritization. relatively large amounts of frequently used static data can be effectively stored in any of these containers.
disk files serve as an emergency copy of data. use them when you don't need or can't afford to keep all the data in memory, but when going to the database is too costly. finally, dbms views are just like virtual tables that represent the data from one or more tables in an alternative way. views are normally used for read-only data, but under certain conditions they can be updateable.
views can also be used as a security mechanism to restrict the data that a certain user can access. for example, some data can be available to users for query and/or update purposes, while the rest of the table remains invisible. and table views can constitute an intermediate storage for preprocessed or post-processed data. therefore, accessing a view has the same effect for the application, but doesn't cause preprocessing delays or place any locks on the physical table.
xml server-side data islands
caching is particularly useful when you have a large amount of data to load. however, when the amount of data is really huge, any technique—either on the client or the server—can hardly be optimal. when you have one million records to fetch, you're out of luck. in such situations, you can reduce the impact of the data bulk by using a layered architecture for caching by bringing the concept of client-side data islands to the server. an xml data island is a block of xml that is embedded in html and can be retrieved through the page's dom. they're good at storing read-only information on the client, saving round-trips.
used on the server, an xml data island becomes a persistent bulk of information that you can store in memory, or (for scalability) on disk. but, how do you read it back? typically, in .net you would use dataset xml facilities to read and write. for lots of data (say, one million records), caching this way is not effective if you don't need all records in memory. keeping all the records in a single file makes it heavier for the system. what about splitting records into different xml files that are organized like those in figure 4? this expands the level of xml disk files shown in figure 3.
figure 4 dividing records for performance
you can build up an extensible tree of xml files, each representing a page of database records. each time you need a block of non-cached records, you fetch them from the database and add them to a new or existing xml data island. you would use a special naming convention to distinguish files on a per-session basis, for example, by appending a progressive index to the session id. an index file can help you locate the right data island where a piece of data is cached. for really huge bulks of data, this minimizes the processing on all tiers. however, with one million records to manage there is no perfect tool or approach.
automatic cache bubble-up
once you have a layered caching system, how you move data from one tier to the next is up to you. however, asp.net provides a facility that can involve both a disk file and a cache object. the cache object works like an application-wide repository for data and objects. cache looks quite different from the plain old application object. for one thing, it is thread-safe and does not require locks on the repository prior to reading or writing.
some of the items stored in the cache can be bound to the timestamp of one or more files or directories as well as an array of other cached items. when any of these resources change, the cached object becomes obsolete and is removed from the cache. by using a proper try/catch block you can catch the empty item and refresh the cache.
string strfile;
strfile = server.mappath(session.sessionid + ".xml");
cachedependency fd = new cachedependency(strfile);
dataset ds = deserializeddatasource();
cache.insert("mydataset", ds, fd);
to help the scavenging routines of the cache object, you can assign some of your cache items with a priority and even a decay factor that lowers the priority of the keys that have limited use. when working with the cache object, you should never assume that an item is there when you need it. always be ready to handle an exception due to null or invalid values. if your application needs to be notified of an item's removal, then register for the cache's onremove event by creating an instance of the cacheitemremovedcallback delegate and passing it to the cache's insert or add method.
cacheitemremovedcallback onremove = new
cacheitemremovedcallback(dosomething);
the signature of the event handler looks like this:
void dosomething(string key, object value,
cacheitemremovedreason reason)
from dataset to xml
when stored in memory, the dataset is represented through a custom binary structure like any .net class. each and every data row is bound with two arrays: one for the current value and one for the original value. the dataset is not kept in memory as xml, but xml is used for output when the dataset is remoted through app domains and networks or serialized to disk. the xml representation of a dataset object is based on diffgrams—a subset of the sql server™ 2000 updategrams. it is an optimized xml schema that describes changes the object has undergone since it was created or since the last time changes were committed.
if the dataset—or any contained datatable and datarow object—has no changes pending, then the xml representation is a description of the child tables. if there are changes pending, then the remoted and serialized xml representation of the dataset is the diffgram. the structure of a diffgram is shown in figure 5. it is based on two nodes, <before> and <after>. a <before> node describes the original state of the record, while <after> exposes the contents of the modified record. an empty <before> node means the record has been added and an empty <after> node means the node has been deleted.
the method that returns the current xml format is getxml, which returns a string. writexml saves the content to a stream while readxml rebuilds a living instance of the dataset object. if you want to save a dataset to xml, use writexml directly (instead of getting the text through getxml) then save using file classes. when using writexml and readxml, you can control how data is written and read. you can choose between the diffgram and the basic format and decide if the schema information should be saved or not.
working with paged data sources
there is a subtler reason that makes caching vital in asp.net. asp.net relies heavily on postback events, so when posted back to the server for update, any page must rebuild a consistent state. each control saves a portion of its internal state to the page's view state bag. this information travels back and forth as part of the html. asp.net can restore this information when the postback event is processed on the web server. but what about the rest? let's consider the datagrid control.
the datagrid gets its contents through the datasource property. in most cases, this content is a datatable. the grid control does not store this potentially large block of data to the page's view bag. so, you need to retrieve the datatable each time a postback event fires, and whenever a new grid page is requested per view. if you don't cache data, you're at risk. you repeatedly download all the data—say, hundreds of records—just to display the few that fit into the single grid page. if data is cached, you significantly reduce this overhead. this said, custom paging is probably the optimal approach for improving the overall performance of pagination. i covered the datagrid custom paging in the april 2001 issue. although that code was based on beta 1, the key points apply. i'll review some of them here.
to enable custom pagination, you must set both the allowpaging and allowcustompaging properties to true. you can do that declaratively or programmatically. next, you arrange your code for pagination as usual and define a proper event handler for pageindexchanged. the difference between custom and default pagination for a datagrid control is that when custom paging is enabled, the control assumes that all the elements currently stored in its items collection—the content of the object bound to the datasource property—are part of the current page. it does not even attempt to extract a subset of records based on the page index and the page size. with custom paging, the programmer is responsible for providing the right content when a new page is requested. once again, caching improves performance and scalability. the caching architecture is mostly application-specific, but i consider caching and custom pagination vital for a data-driven application.
data readers
to gain scalability i'd always consider caching. however, there might be circumstances (such as highly volatile tables) in which project requirements lead you to consider alternative approaches. if you opt for getting data each time you need it, then you should use the datareader classes instead of datasets. a datareader class is filled and returned by command classes like sqlcommand and oledbcommand. datareaders act like read-only, firehose cursors. they work connected, and to be lightweight they never cache a single byte of data. datareader classes are extremely lean and are ideal for reading small portions of data frequently. starting with beta 2, a datareader object can be assigned to the datasource property of a datagrid, or to any data-bound control.
by combining datareaders with the grid's custom pagination, and both with an appropriate query command that loads only the necessary portions of records for a given page, you can obtain a good mix that enhances scalability and performance. figure 6 illustrates some c# asp.net code that uses custom pagination and data readers.
as mentioned earlier, a datareader works while connected, and while the reader is open, the attached connection results in busy. it's clear that this is the price to pay for getting up-to-date rows and to keep the web server's memory free. to avoid the overturn of the expected results, the connection must be released as soon as possible. this can happen only if you code it explicitly. the procedure that performs data access ends as follows:
conn.open();
dr = cmd.executereader(commandbehavior.closeconnection);
return dr;
you open the connection, execute the command, and return an open datareader object. when the grid is going to move to a new page, the code looks like this:
grid.datasource = createdatasource(grid.currentpageindex);
grid.databind();
dr.close();
once the grid has been refreshed (databind does that), explicitly closing the reader is key, not only to preserve scalability, but also to prevent the application's collapse. under normal conditions, closing the datareader does not guarantee that the connection will be closed. so do that explicitly through the connection's close or the dispose method. you could synchronize reader and connection by assigning the reader a particular command behavior, like so:
dr = cmd.executereader(commandbehavior.closeconnection);
in this way, the reader enables an internal flag that automatically leads to closing the associated connection when the reader itself gets closed.
sql statements
the standards of the sql language do not provide special support for pagination. records can be retrieved only by condition and according to the values of their fields, not based on absolute or relative positions. retrieving records by position—for example, the second group of 20 records in an sorted table—can be simulated in various ways. for instance, you could use an existing or custom field that contains a regular series of values (such as 1-2-3-4) and guarantee its content to stay consistent across deletions and updates. alternatively, you could use a stored procedure made of a sequence of select statements that, through sorting and temporary tables, reduces the number of records returned from a particular subset. this is outlined in this pseudo sql:
— first n records are, in reverse order, what you need
select into tmp top page*size field_names
from table order by field_name desc
— only the first "size" records are, in reverse order,
— copied in a temp table
select into tmp1 top size field_names from tmp
— the records are reversed and returned
select field_names from tmp1 order by field_name desc
you could also consider t-sql cursors for this, but normally server cursors are the option to choose when you have no other option left. the previous sql code could be optimized to do without temporary tables which, in a session-oriented scenario, could create serious management issues as you have to continuously create and destroy them while ensuring unique names.
more efficient sql can be written if you omit the requirement of performing random access to a given page. if you allow only moving to the next or previous page, and assume to know the last and the first key of the current page, then the sql code is simpler and faster.
conclusion
caching was already a key technique in asp, but it's even more important in asp.net—not just because asp.net provides better infrastructural support for it, but because of the architecture of the web forms model. a lot of natural postback events, along with a programming style that transmits a false sense of total statefulness, could lead you to bad design choices like repeatedly reloading the whole dataset just to show a refreshed page. to make design even trickier, many examples apply programming styles that are only safe in applications whose ultimate goal is not directly concerned with pagination or caching.
the take-home message is that you should always try to cache data on the server. the session object has been significantly improved with asp.net and tuned to work in most common programming scenarios. in addition, the cache object provides you with a flexible, dynamic, and efficient caching mechanism. and remember, if you can't afford caching, custom paging is a good way to improve your applications.
新闻热点
疑难解答
图片精选
