Table of Contents

• Research Summary
• Selected Recent Publications
• Courses Taught

Research Interests

See also the home page of my former and current graduate students:

• Ayhan Demiriz.
• Jinbo Bi .
• Michinari Momma .

Interested in learning more about Support Vector Machines, Here are the slides from an SVM overview talk in powerpoint, Support Vector Machines: Hype or Hallelujah? and the article from SIGKDD explorations

If you would like to learn about our feature selection and visualization methodology for SVM see the talk "Dimensionality Reduction via Sparse Support Vector Machines given at the NIPS 2001 Workshop on Feature Selection.

Here is more info on our project on automated drug discovery via data mining .

Selected Recent Publications (a bit behind)--

• G. Kunapuli, K. P. Bennett, J. Hu, and J.-S. Pang, “Bilevel Model Selection for Support Vector Machines", Data Mining and Mathematical Programming, P. M. Pardalos and P. Hansen, CRM Proceedings and Lecture Notes, American Mathematical Society, volumne 45, 2008, in press.
• C. Bergeron, T. Hepburn, M. Sundling, N. Sukumar, K. P. Bennett, C. Breneman, “Prediction of bonding affinity in peptides: using kernel methods for nonlinear modeling,” Protein and Peptide Letters, to appear 2008. Documents winning entry of 2006 COEPRA Contest.

J. Hu, J. E. Mitchell, J.-S. Pang, K. P. Bennett and G. Kunapuli. On the Global Solution of Linear Programs with Linear Complementarity Constraints. SIAM Journal on Optimization, to appear, 2008.

• K. P. Bennett, “Discussion: Evidence Contrary to the Statistical View of Boosting by David Mease and Abraham Wynar”, Journal of Machine Learning Research, to appear 2008.
• A. Demiriz, K. P. Bennett, and P. S. Bradley, “Using Assignment Constraints to Avoid Empty Clusters in k-means Clustering”. Constrained Clustering: Advances in Algorithms, Theory, and Applications, S. Basu, I. Davidson, and K. Wagstaff, CRC Press, pg. 203-219, 2008.
• G. Kunapuli, K. P. Bennett, J. Hu, and J.-S. Pang, “Classification Model Selection via Bilevel Programming,” Computational Optimization and Applications, to appear 2008.
• K. P. Bennett, C. Bergeron, E. Acar, R. Klees. S. Vandenberg, B. Yener, and G. Plopper, "Proteomics reveals multiple routes to the osteogenic phenotype in mesenchymal stem cells”, BMC Genomics, 8:380, 2007.
• R. Klees, R. Salasznyk, S. Vandenberg, K. Bennett, and G. Plopper, "Laminin-5 Activates Extracellular Matrix Production and Osteogenic Gene Focusing in Human Mesenchymal Stem Cells", Matrix Biology, 2007.
• P. Agius, B. Kreiswirth, N. Kurepina, and K. P. Bennett, Typing Staphylococcus aureus using the spa gene and novel distance measures, IEEE Transactions on Computational Biology and Bioinformatics, 2007.
• K. P. Bennett, J. Hu, G. Kunapuli, and J.-S. Pang, "Model Selection via Bilevel Optimization", International Joint Conference in Neural Networks, Vancouer 2006.
• The Interplay of Optimization and Machine Learning Research, the introduction to the JMLR Special Topic on Machine Learning and Large Scale Optimization, edited by K. P. Bennett and Emilio Parrado-Hernandez, 2006.
• Inna Vitol, Jeffrey Driscoll, Barry Kreiswirth, Natalia Kurepina, Kristin P. Bennett, " Identifying Mycobacterium tuberculosis Complex Strain Families using Spoligotypes", Infection, Genetics, and Evolution, Nov;6(6):491-504, 2006. The SPOTCLUST program that goes with this can be found at www.rpi.edu/~bennek/EpiResearch.
• Salasznyk, R.M., R. F. Klees, S.Vandenberg, S., K. P. Bennett, and G.E. Plopper, Gene focusing as a basis for controlling stem cell differentiation. Stem Cells and Development, 14(6)608-620, 2005. Note this was considered a "Defining Report" and was featured in the editorial for that issue.
• Salasznyk, R.M., R. F. Klees, S. Vandenberg, S., K. Bennett and G. E. Plopper, Protein expression profiling using gene ontologies of human mesenchymal stem cells during osteogenic differentiation induced by ascorbic acid-2-phosphate, ?-glycerophosphate, and dexamethasone. Stem Cells and Development, 14(4):354-66, 2005.
• M. Momma and K. Bennett, "Constructing Orthogonal Latent Features for Arbitrary Loss", Feature Extraction, Foundations and Applications, Isabelle Guyon, Steve Gunn, Masoud Nikravesh, and Lofti Zadeh, editors, Springer, 2006. accepted 2004.
• M. Fukunari, K. P. Bennett and C. Malmborg, “Decision-Tree Learning in Dwell Point Policies in Autonomous Vehicle Storage and Retrieval Systems (AVSRS)”, International Conference on Machine Learning and Applications 2005.
• A. Malipani, Y.-F. Huang, S. Andra, and K. P. Bennett, “Kernelized Set-Membership Approach to Adaptive Filtering”, 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005.
• Jinbo Bi, T. Zhang and K. P. Bennett, “Column-Generation Boosting Methods for Mixture of Kernels,” Proceedings of SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, 2004, Joydeep Ghosh, editor, ACM Press, 2004.
• N. Tugcu, M. Song, C. Breneman, N. Sukumar, K. P. Bennett, and S. Cramer, “Prediction of the effect of mobile-phase salt type on protein retention and selectivity in anion exchange systems”, Analytical Chemistry, 75:14, 2004, pp. 3563-3572.
• "Regression Error Characteristic Curves ", Jinbo Bi and K. P. Bennett, Proceedings of the 20th International Conference on Machine Learning, 2003.
• "A Geometric Approach to Support Vector Regression", Jinbo Bi and K. P. Bennett, Neurocomputing, 55, 2003, pp. 79-108.

• Sparse Kernel Partial Least Squares Regression.
  M. Momma and K.P. Bennett, Proceedings of Conference on Learning Theory, 2003.

• K. P. Bennett and M. J. Embrechts, "An Optimization Perspective on Partial Least Squares", in J.A.K. Suykens, G. Horvath, S. Basu, C. Micchelli, J. Vandewalle (Eds.) Advances in Learning Theory: Methods, Models and Applications, NATO Science Series III: Computer \& Systems Sciences, Volume 190, IOS Press Amsterdam, 2003,p. 227-250.

• J. Bi, K. Bennett, M. Embrechts, C. Breneman, and M. Song, "Dimensionality Reduction via Sparse Support Vector Machines", Journal of Machine Learning Research, 3, 2003, 1229-1243.

• M. Song, C. Breneman, J. Bi, N. Sukumar, K. Bennett, S. Cramer, and N. Tugcu, "Prediction of Protein Retention Times in Anion-Exchange Chromatography Systems Using Support Vector Regression", September 2002, Journal of Chemical Information and Computer Sciences.

• K. Bennett, M. Momma, and J. Embrechts, MARK: A Boosting Algorithm for Heterogeneous Kernel Models, Proceedings of SIGKDD International Conference on Knowledge Discovery and Data Mining, 2002.

• "Exploiting Unlabeled Data in Ensemble Methods", with A. Demiriz, and R. Maclin, to appear in Proceedings of SIGKDD International Conference on Knowledge Discovery and Data Mining, 2002. \A> >

• C. Breneman, K. Bennett, M. Embrechts, S. Cramer, M. Song, and J. Bi, "Descriptor Generation, Selection and Model Building in Quantitative Structure-Property Analysis", in Chapter 11 of Experimental Design for Combinatorial and High Throughput Materials Development, J. Crawse editor, Wiley, 2002.

• "A Pattern Search Method for Model Selection of Support Vector Regression with M. Momma, to appear in Proceedings of SIAM Conference on Data Mining, 2002.

• "Duality, Geometry, and Support Vector Regression with Jinbo Bi, June 2001, Advances in Neural Information Processing Systems 14, T. Dietterich, S. Becker and Z. Ghahramani editors, MIT Press, Cambridge, pg 593-600. 2002.

• Support Vector Machines: Hype or Hallelujah? with Colin Campbell, SIGKDD Explorations, 2,2, 2000, 1-13.

• Support Vector Machine Regression in Chemometrics” with A. Demiriz, C. Breneman, M. Embrechts, “ Computing Science and Statistics, 2001

• Sparse regression ensembles in infinite and finite hypothesis spaces, with Gunnar Raetsch and Ayhan Demiriz, Machine Learning, 48, 1-3, 2002, pp 193-221.

• Linear Programming Boosting via Column Generation, with Ayhan Demiriz and John Shawe-Taylor, Machine Learning, 46:1, 2001, 225-254.

• Constrained K-Means Clustering, with Paul Bradley and Ayhan Demiriz, Microsoft Research Technical Report 2000-65, May 2000.

• A Column Generation Algorithm for Boosting , with Ayhan Demiriz and John Shawe-Taylor, Proceedings of the Seventeenth International Conference on Machine Learning, Pat Langley Editor, Morgan Kaufmann, San Francisco, 2000, pp. 65-72.

• Duality and Geometry in SVM Classifiers, with Erin Bredensteiner, Proceedings of the Seventeenth International Conference on Machine Learning, Pat Langley Editor, Morgan Kaufmann, San Francisco, 2000, pp. 57-64.

• Large Margin Trees for Induction and Transduction, with Donghui Wu, Nello Cristianini, and John Shawe-Taylor, ICML'99 Proceedings.

• Density-Based Indexing for Approximate Nearest Neighbor Queries, with Usama Fayyad and Dan Geiger, Microsoft Research Technical Report 98-58, Microsoft Research, Redmond WA, 19998. Longer version of paper in KDD'99 Proceedings.

• Enlarging Margins in Perceptron Decision Trees , with Donghui Wu, Nello Cristianini, and John Shawe-Taylor Machine Learning, 41:3, 295-313.

• Semi-Supervised Clustering Using Genetic Algorithms, with Ayhan Demiriz and Mark Embrechts, ANNIE'99 (Artificial Neural Networks in Engineering), November 1999. Long version: Genetic Algorithm Approach for Semi-supervised Clustering is under review for journal publication.

• Optimization Approaches to Semi-Supervised Learning , with Ayhan Demiriz, to appear in Applications and Algorithms of Complementarity, M. C. Ferris, O. L. Mangasarian, and J. S. Pang, editors. Kluwer Academic Publishers. Boston 2000.

• Multicategory Classification by Support Vector Machines. with E. J. Bredensteiner, Computational Optimizations and Applications, 12, 1999, pp 53-79.

• On Mathematical Programming Methods and Support Vector Machines, in Advances in Kernel Methods -- Support Vector Machines, A. Schoelkopf, C. Burges, and A. Smola, editors, MIT Press, Cambridge, MA, 1999, pp 307-326.

• Semi-Supervised Support Vector Machines with A. Demiriz, Advances in Neural Information Processing Systems, 12, M. S. Kearns, S. A. Solla, D. A. Cohn, editors, MIT Press, Cambridge, MA, 1998, pp 368-374.

• On Support Vector Decision Trees for Database Marketing with D. H. Wu and L. Auslender, R.P.I Math Report No. 98-100, Rensselaer Polytechnic Institute, Troy, NY, 1998. Long version of IJCNN'99 paper

• A Support Vector Machine Approach to Decision Trees with J. Blue, R.P.I Math Report No. 97-100, Rensselaer Polytechnic Institute, Troy, NY, 1997.

• Optimal Decision Trees with J. Blue R.P.I Math Report No. 214, 1996.

• Geometry in Learning with E. Bredensteiner, Web Manuscript http:\\www.rpi.edu\~bennek\geometry2.ps September 1996. also in Geometry at Work, C. Gorini editors, Mathematical Association of America, Washington, D.C, 2000, 132-145.

• Hybrid Extreme Point Tabu Search with J. Blue, R.P.I Math Report No. 240, 1996, appeared in European Journal of Operation Research 106, 1998, 676-688

• An Extreme Point Tabu Search Method for Data Mining. with J. Blue, R.P.I Math Report No. 228, 1996.

• Feature Minimization within Decision Trees. with E. Bredensteiner, Computational Optimizations and Applications, 10:2, 1998. 111-126

• A Parametric Optimization Method for Machine Learning. with E. J. Bredensteiner, R.P.I Math Report No. 217, 1995. Final version appeared in INFORMS Journal on Computing, 9:3, pp. 311-318, 1997.

• Global Tree Optimization: A Non-Greedy Decision Tree Algorithm. Computing Science and Statistics, 26, pp. 156-160, 1994

• Serial and Parallel Multicategory Discrimination, SIAM Journal on Optimization, with O. L. Mangasarian, 4:4, pp. 722-734, 1994. Also appeared as Tech Report 1165, Department of Computer Sciences, University of Wisconsin Madition, 1994.

• Machine Learning via Mathematical Programming, Ph.D. Thesis, Department of Computer Sciences, University of Wisconsin Madition, 1994.

• Multicategory Discrimination via Linear Programming, with O. L. Mangasarian, Optimization Methods and Software 3, pp. 27-39, 1993.

• Bilinear Separation of Two Sets in n-Space, with O. L. Mangasarian, Computational Optimization and Applications, 2, pp. 207-227, 1993.

• Decision Tree Construction Via Linear Programming, Proceedings of the 4th Midwest Artificial Intelligence and Cognitive Science Society Conference, Utica, Illinois, pp. 97-101, 1992.

• Robust Linear Programming Discrimination of Two Linearly Inseparable Sets. with O. L. Mangasarian, Optimization Methods and Software, 1, pp. 23-34, 1992.

• Neural Network Training Via Linear Programming, with O. L. Mangasarian, in P. M. Pardalos (ed.), Advances in Optimization and Parallel Computing, pp. 56-67, 1992.