Data integration and management is a particularly broad subject, covering up to ten different areas: data governance; data architecture, analysis, and design; database management; data security management; data quality management; reference and master data management; data warehousing and business intelligence management; document record and content management; meta data management; contact data management. While a number of these areas are specific technologies disciplines, these deal with how data is stored, accessed, utilized, and analyzed.
Data integration involves combining data residing in different sources and providing users with a unified view of these data. Data integration appears with increasing frequency as the volume and the need to share existing data explodes. Data integration can be accomplished in a number of different ways of varying complexity. One solution is based on data warehousing. The warehouse system extracts, transforms, and loads data from heterogeneous sources into a single common queriable schema so data becomes compatible with each other. This system can be searched very quickly, but problems arise with the "freshness" of data, because when an original data source gets updated, the warehouse still retains outdated data and the ETL process needs re-execution for synchronization.
Another process for data integration is a unified query-interface to access real time data over a mediated schema, which allows information to be retrieved directly from original databases. This approach may need to specify mappings between the mediated schema and the schema of original sources, and transform a query into specialized queries to match the schema of the original databases. Searches can be slower in this method, but benefit from having direct access to particular databases as they are updated.
Data management and integration derives value from creating a central repository of information that can be searched and utilized by multiple people at once. Data is only useful when the data has been properly cleaned and normalized, the process of data integration must be combined with best practices in data quality management and database architecture. Data integration and management also takes value from the systems that are built on top of it. The larger the scope of data, the better information a system such as business intelligence has to work with, allowing for a greater breadth of analyses to be completed.
Iknow has deep expertise in all aspects of data integration and management from initial requirements gathering to design of database architecture, to data cleansing, to design of mediated schema, to system customization and deployment. Iknow has experience to integrate databases into other products such as business intelligence, customer intelligence, expert systems, and data mining. We have experience managing data management and integration projects of many different sizes, from creating and designing small databases, to integrating hundreds of databases with a web query that can search all databases at once.