although there are various approaches to data mining that seem to offer distinct features and benefits, many may not be powerful enough to meet your corporate knowledge discovery needs. but in fact just a few fundamental questions can quickly clarify the business benefits and the power of a data mining system, setting its advantages in a clear perspective. these questions need to be asked both from the view points of business and technical users. however, please note that these questions refer to data mining -- please also see the many benefits of the knowledge access paradigm which uses the patterns discovered by data mining within a patternwarehousetm. here are two sets of "top ten data mining questions" from business and technical perspectives. each question has three parts that together highlight one specific aspect of a data mining system's power and capability. the top ten data mining business questions the top ten business question should be asked by business users about the benefits, quality and usability of the system. they are:question 1: business benefits a) how will this system help us? b) how well does this system work for our industry-specific applications? c) what information can we get that we do not already have?it is essential to ask this question again and again. you should, of course, get new refined information, but it is not enough just to know something -- you should have information that allows you to "act" within the context of your industry. and, you should measure the bottom-line dollar benefits delivered by a data mining system. see the paper "measuring the dollar value f mined information" for a framework for this.question 2: technical know-how a) how technically sophisticated do we need to be to use it? b) can business users operate it without calling the is group all the time? c) is it as easy to use as an internet browser?business users should be empowered with direct, on-demand access to refined knowledge. they should not have to know statistics, yet should be given consistent and correct answers. the system interface should be as easy to use as a web-browser.question 3: understandability and explanations a) are the results intuitive or difficult to understand? b) do we get clear explanations for any information item presented? c) will the explanations be in technical statistical terms or in a form that we can understand?results should be presented to business users in plain english, accompanied with graphs. the system should be able to explain each piece of information it presents in clear, english-like terms that business users can easily comprehend and use.question 4: follow-up questions a) what kinds of follow-up questions can we ask from the system? b) do we need to go to an analyst for further question answering? c) how fast can we drill-down on the fly to see more patterns?response to follow-up questions must be immediate. business users should not need to use intermediaries such as analysts to get more information after they have seen some results. if follow-up questions take time and involve intermediaries, the business users effectiveness will be impacted. business users should get refined information, as they need it, when they need it.question 5: business users a) how many business users can this system support? b) can the business users tailor their own questions for the system? c) can users utilize the knowledge for day-to-day decision making?the system should be able to use the same fundamental knowledge to support a few hundred business users, each with a different group-perspective. yet, all of these users must be given consistent answers as they ask their own questions. the information must be presented such that can be utilized for day-to-day actions.question 6: accuracy, completeness and consistency a) how accurate are the results the system delivers? b) can some patterns be missed by the system? c) are the results always consistent or can 100 users get 100 different answers?the system must cover a wide range of patterns and should provide high quality, information. the knowledge provided to business users should be derived from the entire data set (and not samples) in order to increase accuracy. all business users should access the same knowledge so that they all receive consistent answers, increasing the quality of corporate information.question 7: incremental analysis a) can we automatically analyze weekly / monthly data as it becomes available? b) can the system compare the "month to month" results and patterns by itself? c) can we get automatic pattern detection over time, every week or month?the system should analyze data as it becomes available every week or month and perform on-going trend analysis, highlighting the key items and influence factors that impact significant changes. the incremental analysis should be performed automatically in the background, informing the user of significant trends and the underlying causes.question 8: data handling a) how much data can the system deal with? b) can it work directly on our database, or do we need to extract data? c) if it works on extracts, how do we know that some patterns are not missed?the system should handle moderate to large volumes of data on a powerful server -- of course, large data volumes should not be expected to be managed on small servers. the system should work directly on the sql database, without extracts so that patterns are not missed and performance is improved.question 9: integration a) how will it integrate into our computing environment? b) will it just work on our existing sql database? c) how easily will the system work on our intranet?the system should run smoothly on existing open server platforms (e.g. unix) and popular dbms engines (e.g. oracle, sybase informix, etc.) on the server. the system should present results to users on the corporate intranet. the absence of data conditioning requirements and extract files will make integration much easier.question 10: support staff a) what staff do i need to keep this system installed and running? b) how do we get support and training to get started? c) what happens after we install the system?after the initial system design, the support personnel for the system should be kept minimal. one database administrator should be able to manage the dbms, and one analyst should occasionally help in setting up discovery models, etc. thereafter, business users should be able to use the system on their own. there should be no need for a large number of resident support analyst to act as intermediaries for the business users. the top ten data mining technical questionsthe top ten technical question should be asked by technical users about the architecture, power and the scalability of the system. they are:question 1: architecture a) how are computations distributed between the client and the server? b) is any data brought from the server to the client? c) can the system run in a three tiered architecture?the best option is for the discovery to take place entirely on the server. any attempt to bring data to the client will seriously limit the applicability of the system to larger databases. the best architecture is a thin-client, three-tiered system that uses the power of a large server-based sql engine but operates on an intranet. question 2: access to real data a) does the system work on the real sql database or on samples and extracts? b) if it samples or extracts, how do we know that it is accurate? c) if it builds flat files, who manages this activity and cleans up for on-going analyses, and how can it sample across several tables?the best option is for a data mining system to work on the real databases and not on samples, extracts and/or flat files. working on the real database uses the sql engine's power (e.g. parallel execution) and provide much more accurate results. and, the system should be able to access database tables in their native form, reaching across tables by itself.question 3: performance and scalability a) how large of a database can the system analyze? b) how long does it take to perform discovery on a large database? c) can the system run in parallel on a multi-processor server?the system should work on databases with a large number of records. it should derive its capabilities from the power of the server and the sql engine, whenever possible. the system should be able to use the built-in parallelism of the sql engine, but should also be able to use multiple processors for its own parallel non-sql computations.question 4: multi-table databases a) does the system work on a single table only or can it analyze multiple tables? b) does the system need to perform a huge join to access all of our tables? c) if it works on a single table, how can we feed it our existing data schema? the real world is full of multi-table databases which can not be joined and meshed into a single view. in fact, the theory of normalization came about because data needs to be in more than one table. using single tables is an affront to a decade of work on database design. if you challenge the dba of a really large database to put things in a single table you will either get a laugh or a blank stare -- in many cases the database size will balloon beyond control. the system should be able to mine large multi-table databases directly by itself on the server. question 5: multi-dimensional analysis a) does the system analyze data along a single dimension only? b) how are multi-dimensional patterns discovered and expressed by the system? c) how do we specify the dimensional structure of our data to the system?the olap phenomenon has conclusively demonstrated that the business world's data is not single-dimensional. hence a data mining system should be able to automatically discover patterns along multiple dimensions. in fact, there are many cases where no single dimensional view can correctly represent the semantics of influence because the influence ratios will always be off regardless of how one aggregates. see the paper: olap & data mining: bridging the gap for a detailed discussion of this. question 6: types and classes of patterns discovered a) how powerful and general are the patterns the system can discover and express? b) can the system mix different pattern types, e.g. influence and affinity patterns? c) can the system discover time-based patterns and trends?the format of the patterns discovered by the system is very general and goes far beyond decision trees or simple affinities. the advantage to this is that the general rules discovered are far more powerful than decision trees. decision trees are very limited in that they cannot find all the information in a database. being rule-based keeps the system from being constrained to one part of a search space and makes sure that many more clusters and patterns are found -- allowing the system to provide more information and better predictions.question 7: system initiative a) does the system use its own initiative to perform discovery or is it guided by the user? b) can the system discover unexpected patterns by itself? c) can the system start-up by itself on a weekly or monthly basis and perform discovery?in some cases the user has to interact and guide the system, e.g. build a decision tree. however, a better approach is for the system to use its own initiative in the data mining process, forming hypothesis automatically based on the character of the data. the system should start-up by itself, select the significant patterns in the data and filter the unimportant trends. the analyses should be done routinely on a weekly or monthly basis.question 8: treatment of data types a) are all data types handled in their own form or translated to other types? b) can the system find numeric ranges in data by itself? c) do a large number of non-numeric values cause problems for the system?the system should manage all data types in a uniform manner and in their native formats, i.e. numbers, dates and constants should remain numbers, dates and constants internally. interesting ranges in the data should be discovered by the system, not requiring "number bin" construction by the user. a large number of constant values in the database should not choke the system.question 9: data dependencies and hierarchies a) can the system be told about the functional dependencies in our database? b) does the system understand the concept of data hierarchy? c) how does the system use dependencies and/or hierarchies for discovery?the system should be capable of using the functional (and other dependencies) that exist in a database. the use of these dependencies can significantly enhance the power of a discovery -- in fact ignoring them can lead to confusion. the system should understand the concept of hierarchy and should be able to use it for discovery along multiple dimensions.question 10: flexibility and noise sensitivity a) how brittle is the system when dealing with noisy data? b) how well does the system cope with data exceptions and low quality data? c) can the system provide statements with flexible numeric ranges discovered by itself in the data?the system should not be sensitive to noise and should internally use fuzzy logic to smooth data brittleness. as the data gathers noise, the system should only reduce the level on confidence associated with the results provided, not suddenly change direction in discovery. however, the system should still produce the most significant findings from the data set, even if noise is present.
