Data mining, a relatively young and interdisciplinary field of computer science is the process of discovering new patterns from large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics and database systems. The overall goal of the data mining process is to extract knowledge from a data set into a human-understandable structure.
The actual data mining task is the automatic or semi-automatic analysis of large quantities of data to extract previously unknown interesting patterns such as groups of data records, unusual records and dependencies. This usually involves using database techniques such as spatial indexes. These patterns can then be seen as a kind of summary of the input data, and used in further analysis or for example in machine learning and predictive analytics. For example, the data mining step might identify multiple groups in the data, which can then be used to obtain more accurate prediction results by a decision support system. Neither the data collection, the data preparation nor the result interpretation are part of the data mining step, but do belong to the overall process as additional steps.
Data mining's value comes from the ability to easily obtain timely and accurate information for decision-making purposes. Accessing information from these reporting environments often range from ad hoc queries to multidimensional analysis. While these forms of data analysis are excellent at answering the question or questions posed by the user, they do not provide any other insight. Individuals and organizations are recognizing that additional value may lie within the vast amounts of data that they are storing. By applying data mining techniques, which are elements of statistics, artificial intelligence and machine learning, they are able to identify trends within the data that they did not know existed. These techniques can be used for fraud detection, human resources, inventory logistics, defect analysis, business, and competitor and supplier intelligence.
Iknow has deep expertise in all aspects of data mining projects from initial requirements gathering, to database and data management aspects, to model and inference considerations, to post-processing of found structure, to creation of reporting dashboards, visualization and online updating, to system customization and deployment. We have experience managing data mining projects of many different sizes, from a small company trying to understand its own processes, to a large enterprise collecting and analyzing data over an entire competitive ecosystem.