Development of New Data Processing Model for Data and Software Reusability

This thesis studies the challenges in improving the quality of data processing in information intensive software applications, particularly in the practices of software engineering involved in designing and implementing a system model for the various needs of information processing. The research builds upon the traditional methods found in the software engineering literature to create a new data design pattern style for a new data environment. At the same time, as the amount of data keeps growing, the logical management of data (and the various data sources) is becoming more important. Prior research has shown that there is a need for maintainable and systematic ways to manage the produced data more efficiently. How can and should data be managed by the software? To address the aforementioned question, a comprehensive solution will need to apply the disciplines of data management and software engineering together.

The research in this thesis integrates the aforementioned features together in one package, which includes features from software frameworks, design patterns, and architectural styles. The research adopts Design Science Research Methodology in carrying out the research activities, and iteratively refines and evaluates the intermediate results.

The main contribution is the introduction of a conceptual and generic data processing model, which is built on the metaphor of a streaming water apparatus consisting of faucets, sinks, and drains. The main point of the model is how the model treats all data sources equally, and as simply and generically as possible. The generic data source management of the model will be the key on improving the reuse of source code as well as reuse of data. The secondary contribution is a solution derived from the model which provides a reference architecture, definitions, and specifications to realize a generic and reusable software framework.

The work is validated through several architectural iterations and prototype implementations. The framework was implemented and tested in an experimental prototype system with a few use cases. Finally, the findings, and theoretical and practical prospects of the model are discussed. The demonstrated proof-of-concept experiments indicate that the proposed model is feasible solution for systematically manageable data processing, and can improve the quality and reuse of both software and data.