Acta mathematica scientia, Series B >
MODEL SELECTION METHOD BASED ON MAXIMAL INFORMATION COEFFICIENT OF RESIDUALS
Received date: 2013-02-05
Revised date: 2013-12-20
Online published: 2014-03-20
Supported by
This work was partly supported by National Basic Research Program of China (973 Program, 2011CB707802, 2013CB910200) and National Science Foundation of China (11201466).
The traditional model selection criterions try to make a balance between fitted error and model complexity. Assumptions on the distribution of the response or the noise, which may be misspecified, should be made before using the traditional ones. In this ar-ticle, we give a new model selection criterion, based on the assumption that noise term in the model is independent with explanatory variables, of minimizing the association strength between regression residuals and the response, with fewer assumptions. Maximal Information Coefficient (MIC), a recently proposed dependence measure, captures a wide range of associ-ations, and gives almost the same score to different type of relationships with equal noise, so MIC is used to measure the association strength. Furthermore, partial maximal information coefficient (PMIC) is introduced to capture the association between two variables removing a third controlling random variable. In addition, the definition of general partial relationship is given.
TAN Qiu-Heng , JIANG Hang-Jin , DING Xi-Ming . MODEL SELECTION METHOD BASED ON MAXIMAL INFORMATION COEFFICIENT OF RESIDUALS[J]. Acta mathematica scientia, Series B, 2014 , 34(2) : 579 -592 . DOI: 10.1016/S0252-9602(14)60031-X
[1] Claeskens Gerda, Hjort Nils Lid. Model selection and model averaging. Cambridge Books, 1993
[2] Cook R Dennis, Weisberg Sanford. Residuals and influence in regression. Volume 5. New York: Chapman and Hall, 1982
[3] Cucker Felipe, Zhou Ding Xuan. Learning theory: an approximation theory viewpoint. Volume 24. Cambridge University Press, 2007
[4] Das Jishnu, Mohammed Jaaved, Yu Haiyuan. Genome-scale analysis of interaction dynamics reveals organization
of biological networks. Bioinformatics, 2012, 28(14): 1873–1878
[5] Fan Jianqing, Lv Jinchi. A selective overview of variable selection in high dimensional feature space. Statistica Sinica, 2010, 20(1): 101
[6] Gelfand L M, Jaglom A M, Kolmogoroff A N. Zur allgemeinen definition der information. Arbeiten zur Informationstheorie II. Berlin: VEB Deutscher Verlag der Wissenschaften, 1958
[7] Gorfine Malka, Heller Ruth, Heller Yair . Comment on “detecting novel associations in large data sets”.
Unpublished manuscript. http://iew3. technion. ac. il/˜ gorfinm/files/science6. pdf, 2012
[8] Hastie Trevor, Tibshirani Robert, Friedman J Jerome H. The elements of statistical learning. Volume 1. New York: Springer, 2001
[9] Heller Ruth, Heller Yair, Gorfine Malka. A consistent multivariate test of association based on ranks of distances. arXiv preprint arXiv:1201.3522, 2012
[10] Karpinets Tatiana V, Park Byung H, Uberbacher Edward C. Analyzing large biological datasets with association networks. Nucleic Acids Research, 2012, 40(17): e131
[11] Kolmogoroff Alexander. Grundbegriffe der wahrscheinlichkeitsrechnung. Volume 3. Berlin: Springer, 1933
[12] Konishi Sadanori, Kitagawa Genshiro. Information criteria and statistical modeling. Springer Verlag, 2008
[13] Linfoot E H. An informational measure of correlation. Information and Control, 1957, 1(1): 85–89
[14] NIST/SEMATECH. e-Handbook of Statistical Methods. http://www.itl.nist.gov/div898/handbook/, 2013
[15] R´enyi Alfr´ed. On measures of dependence. Acta Mathematica Hungarica, 1959, 10(3): 441–451
[16] Reshef David N, Reshef Yakir A, Finucane Hilary K, et al. Detecting novel associations in large data sets. Science, 2011, 334(6062): 1518–1524
[17] Shannon Claude Elwood, Weaver Warren. A mathematical theory of communication. Univ of Illinois Press, 1949
[18] Shi Peide, Tsai Chih-Ling. Regression model selection a residual likelihood approach. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2002, 64(2): 237–252
[19] Shumway Robert H , Stoffer David S. Time series analysis and its applications. Springer Science+ Business Media, 2010
[20] Speed Terry. A correlation for the 21st century. Science, 2011, 334(6062): 1502–1503
[21] Steuer Ralf, Kurths J¨urgen, Daub Carstens O, Weise Janko, Selbig Joachim. The mutual information: detecting and evaluating dependencies between variables. Bioinformatics, 2002, 18(suppl 2): S231–S240
[22] Sz´ekely G´abor J, Rizzo Maria L . Brownian distance covariance. The annals of applied statistics, 2009: 1236–1265
[23] Zhang Xiujun, Liu Keqin, Liu Zhi-Ping, et al. Narromi: a noise and redundancy reduction technique improves accuracy of gene regulatory network inference. Bioinformatics, 2013, 29(1): 106–113
/
| 〈 |
|
〉 |