A Multi-Layered Immune Inspired Approach to Data Mining

A Multi-Layered Immune Inspired Approach to Data Mining. Thomas Knight and Jon Timmis Computing Laboratory University of Kent at Canterbury. CT2 7NF. UK e-mail: tpk1, jt6@ukc.ac.uk Keywords: Machine Learning, Artificial Immune Systems, Data Mining. Abstract Soft computing has been described as computational systems that exploit tolerance for imprecision, uncertainty, partial truth and approximation [1]. Examples of which include artificial neural networks, fuzzy systems, evolutionary algorithms and probabilistic reasoning. Artificial immune systems have recently been proposed as an additional soft computing paradigm [2]. In that paper, the authors argue that artificial immune systems exhibit similar characteristics to other soft computing paradigms which can be used to complement and augment existing soft computing techniques. Artificial Immune Systems are adaptive systems, inspired by theoretical immunology and observed immune functions, principles, and models, which are applied to problem solving [3]. An immune inspired data mining algorithm was proposed in [4], called RAIN (Resource-limited Artificial Immune Network). The motivation behind this algorithm was to abstract processes and characteristics from the immune system, more specifically the immune network theory [5] to develop a novel unsupervised machine learning algorithm. RAIN's artificial immune networks were applied to unsupervised machine learning benchmark data and was reported to perform well. It was anticipated that this algorithm would be suitable for continuous learning with initial work showing that stable populations within the networks could be achieved. However, more recent work has since shown that these networks suffer strong evolutionary pressure and converge to the strongest class represented in the data [6]. Whilst this is not perceived as a major problem from a data mining perspective, for use in continuous learning, however, it would be more desirable if the networks converged to a stable population that is representative of all of the classes in the data and these patterns then remained in the absence of antigenic stimulation (training data). These observations prompted a step back from the existing work to re-evaluate the approaches taken. It was noted that a more holistic approach may provide a better solution in the search for an immune inspired data mining algorithm capable of continuous learning. Rather than focusing on the immune network theory, we have adopted aspects of the primary and secondary responses seen in the adaptive immune system. A multi-layered approach has been devised that incorporates interactions between free-antibodies, b-cells, and memory cells using clonal-selection processes as the core element of the algorithm. This three-layered approach consists of: a free-antibody layer, a b-cell layer and a memory layer. The free-antibody layer provides a general search and pattern recognition function. The b-cell layer provides a more refined pattern recognition function, with the memory layer providing a stable memory structure that is no longer influenced by strong evolutionary pressure. Central to the algorithm is feedback that occurs between the b-cell layer and the free-antibody layer: this produces co-stimulation between b-cells and is part of the secondary immune response in the algorithm. Novel data are incorporated into the b-cell layer and is given a chance to thrive, thus providing a primary immune response. Although the algorithm has been designed for continuous learning, initial testing has highlighted good performance at static clustering and is thus reported in this paper. Initial studies of the algorithm suggests that the performance compares favorably to the nearest immune inspired competitor, a combined clonal selection and immune network based algorithm called aiNet [7]. Although the algorithm itself is very simple, it achieves good representation of data (Figure 1), and compression ratios. (a) (b) Figure 1 (a) shows the 3-circle training set (600 items), and (b) shows the patterns in the memory layer evolved by the new algorithm after 5 iterations (58 items) Preliminary results are encouraging and it is thought the proposed algorithm can be adapted for continuous learning. It is also proposed that this algorithm augments the framework for AIS proposed in [2] with the addition of a new immune inspired algorithm. Citations [1] Y. Jin, ''What is Soft Computing,'' in A Definition Of Soft Computing - adapted from L.A. Zadeh, vol. 2002, 2002. [2] L. N. de Castro and J. I. Timmis, ''Artificial Immune Systems as a Novel Soft Computing Paradigm,'' Soft Computing, vol. In Press, 2002. [3] J. Timmis and L. N. De Castro, ''A Framework for Engineering Artificial Immune Systems,'' in Artificial Immune Systems: A New Computational Intelligence Approach: Springer Verlag, 2002, pp. 53-108. [4] J. Timmis and M. Neal, ''A resource limited artificial immune system for data analysis,'' Knowledge Based Systems, vol. 14, pp. 121-130, 2001. [5] N. K. Jerne, ''Towards a Network theory of the Immune System,'' Annals of Immunology, vol. 125C, pp. 373-389, 1974. [6] T. P. Knight and J. I. Timmis, ''AINE: An immunological approach to data mining,'' presented at IEEE International Conference on Data Mining, San Jose, CA. USA, 2001. [7] L. N. de Castro and F. N. Von Zuben, ''An Evolutionary Immune Network for Data Clustering,'' Proceedings of the IEEE Computer Society Press, SBRN'00, vol. 1, pp. 84-89, 2000. Acknowledgements Thomas Knight would like to thank the SUN(tm) Microsystems for their continued financial support for his PhD Studies.