<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <link href="https://bactra.org/plain-style.css" rel="stylesheet" type="text/css"> <title>Data Mining</title>        </head> <body>   <P> <P></P><!-- Access to MathJax through the version I'm hosting on bactra.org !--><!-- but only IF mathmode is activated via meta_mathmode !-->  <script> window.MathJax = {  tex: {    inlineMath: [['$','$'], ['\\(','\\)']],    processEscapes: true,    autoload: {      color: [],      colorv2: ['color']    },    packages: {'[+]': ['noerrors']}  },  options: {    ignoreHtmlClass: 'tex2jax_ignore',    processHtmlClass: 'tex2jax_process'  },  loader: {    load: ['[tex]/noerrors']  }};</script>      <cite><a href="http://bactra.org/notebooks">Notebooks</a></cite> <div class="text"><div class="left"><h2>Data Mining</h2> <i><a href="faq.html#5.4">Last update</a>: 30 Jun 2025 10:59</i><br><i><a href="faq.html#5.4">First version</a>: 24 August 2006</i> <hr> <P>  <P>I've taught a course on this, so I ought to be able to describe it, oughtn'tI?  Data mining, more stuffily "knowledge discovery in databases", is the artof finding and extracting useful patterns in very large collections of data.It's not quite the same as <a href="learning-inference-induction.html">machinelearning</a>, because, while it certainly uses ML techniques, the aim is todirectly guide action (<em>praxis</em>!), rather than to develop a technologyand theory of induction.  In some ways, in fact, it's closer towhat <a href="statistics.html">statistics</a> calls "exploratory dataanalysis", though with certain advantages and limitations that come from havingreally big data to explore. <P><a href="kernel-methods.html">Kernel methods</a> and <a href="nearest-neighbors.html">nearest neighbors</a> get their own notebooks. <P>Ethical and political issues in datamining <a href="ethics-politics-data-mining.html">definitely deservetheir own notebook</a>.  <ul>See also:	<li><a href="clinical-vs-actuarial.html">Clinical and Actuarial Compared</a>	<li><a href="clustering.html">Clustering</a>	<li><a href="neural-nets.html">Neural Nets, Connectionism, Perceptrons, etc.</a>	<li><a href="recommenders.html">Recommender Systems and Collaborative Filtering</a>	<li><a href="structured-data.html">Statistics for Structured Data</a>	<li><a href="text-mining.html">Text Mining</a>	<li><a href="trees.html">(Decision, Classification, Regression, Prediction) Trees in Statistics and Machine Learning</a>	<li><a href="uncertainty-for-neural-networks.html">Uncertainty for Neural Networks, and Other Large Complicated Models</a>	<li><a href="variable-selection.html">Variable and Feature Selection for Regression and Classification</a>	</ul>  <ul>Recommended, big picture:	<li>Leo Breiman, "Statistical Modeling: The Two Cultures",<a href="http://projecteuclid.org/euclid.ss/1009213726"><cite>Statistical Science</cite> <strong>16</strong> (2001): 199--231</a> [verymuch including the discussion by others and the reply by Breiman]	<li>Pedro Domingos, <a href="http://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf">"A Few Useful Things to Know about Machine Learning"</a>	<li>David Hand, Heikki Mannila and Padhraic Smyth, <a href="../weblog/algae-2010-01.html#data-mining"><cite>Principles ofData Mining</cite></a> [The textbook I teach from; also a book I learned a lot from.]	<li>Trever Hastie, Robert Tibshirani and Jerome Friedman, <cite>TheElements of Statistical Learning: Data Mining, Inference, and Prediction</cite>[<a href="http://www-stat.stanford.edu/~tibs/ElemStatLearn/">Website</a>, with full text free in PDF]	<li>Cathy O'Neil, <a href="http://bactra.org/weblog/algae-2019-09.html#oneil-wmd"><cite>Weapons of Math Destruction</cite></a>	<li>Sholom M. Weiss and Nitin Indrukyha, <cite>Predictive DataMining: A Practical Guide</cite> [Pedestrian, but it <em>is</em> practical, andadapted to the meanest, i.e. the managerial, understanding]	</ul> <ul>Recommended, close-ups:	<li>Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan, Christopher Nielson, Thomas F. Osborne, Rajiv Raman, Kim Ramasamy, Rory Sayres, Jessica Schrouff, Martin Seneviratne, Shannon Sequeira, Harini Suresh, Victor Veitch, Max Vladymyrov, Xuezhi Wang, Kellie Webster, Steve Yadlowsky, Taedong Yun, Xiaohua Zhai, D. Sculley,"Underspecification Presents Challenges for Credibility in Modern Machine Learning", <a href="http://arxiv.org/abs/2011.03395">arxiv:2011.03395</a>	<li>Jesse Davis and Mark Goadrich, "The Relationship Between Precision-Recall and ROC Curves" [<a href="http://pages.cs.wisc.edu/~jdavis/davisgoadrichcamera2.pdf">PDF preprint</a>]	<li>Sharad Goel, Jake M. Hofman, S&eacute;bastien Lahaie, David M. Pennock, and Duncan J. Watts, "Predicting consumer behavior with Web search",<a href="https://doi.org/10.1073/pnas.1005962107"><cite>Proceedings of the National Academy of Sciences</cite>(USA) <strong>107</strong> (2010): 17486--17490</a> [A case study in usingdata mining, while recognizing limitations]	<li>Aleks Jakulin and Ivan Bratko, "Quantifying and VisualizingAttribute Interactions", <ahref="http://arxiv.org/abs/cs.AI/0308002">cs.AI/0308002</a>	<li>Jacob Kogan, <cite>Introduction to Clustering Large and High-Dimensional Data,/cite>	<li>Jon Kleinberg, Christos Papadimitriou and Prabhakar Raghavan, "AMicroeconomic View of Data Mining", <cite>Data Mining and KnowledgeDiscovery</cite> <strong>2</strong> (1998)[<a href="http://www.cs.cornell.edu/home/kleinber/dmkd98-seg.pdf">PDF</a>]	<li>Ariel Kleiner, Ameet Talwalkar, Purnamrita Sarkar, Michael I. Jordan, "A Scalable Bootstrap for Massive Data", <a href="http://arxiv.org/abs/1112.5016">arxiv:1112.5016</a>	<li>Kling, Scherson and Allen, "Parallel Computing and InformationCapitalism," in Metropolis and Rota (eds.), <cite>A New Era inComputation</cite> (1992) [A batch of UC Irvine comp. sci. professors who writelike <a href="sociology.html">sociologists</a>.  " `Information capitalism'refers to forms of organization in which data-intensive techniques andcomputerization are key strategic resources for corporate production."]	<li>Jure Leskovec, Anand Rajaraman and Jeffrey David Ullman, <a href="http://www.mmds.org"><cite>Mining of Massive Datasets</cite></a>	<li>R. Dean Malmgren, Jake M. Hofman, Luis A. N. Amaral, Duncan J. Watts, "Characterizing Individual Communication Patterns", <a href="http://arxiv.org/abs/0905.0106">arxiv:0905.0106</a>	<li>John Shawe-Taylor and Nello Cristianini, <cite>Kernel Methodsfor Pattern Analysis</cite>	<li>Ryan J. Tibshirani, "Degrees of Freedom and Model Search", <a href="http://arxiv.org/abs/1402.1920">arxiv:1402.1920</a>	<li>Jianming Ye, "On Measuring and Correcting the Effects of Data Mining and Model Selection", <a href="https://doi.org/10.1080/01621459.1998.10474094"><cite>Journal of the American Statistical Association</cite> <strong>93</strong> (1998): 120--131</a>	</ul>     <ul>Modesty forbids me to recommend:	<li>My <a href="http://www.stat.cmu.edu/~cshalizi/dm/">lecture notesfor my data mining class</a> [I learned a lot, in writing these, from notesfrom the previous version of the course writtenby <a href="http://research.microsoft.com/en-us/um/people/minka/">TomMinka</a>, and modesty does not forbid me from recommending his work.]	</ul> <ul>To read, popular expositions and social and economic consequences:	<li>Ajay Agrawal, Joshua Gans and Avi Goldfarb		<ul>		<li><cite>Prediction Machines: The Simple Economics of Artificial Intelligence</cite>		<li>"AI Adoption and System-Wide Change", <a href="https://doi.org/10.3386/w28811">NBER Working Paper 28811 (2021)</a>		</ul>	<li>Ian Ayres, <cite>Super Crunchers: Why Thinking-by-Numbers Is theNew Way to Be Smart</cite> [Despite the <em>painful</em> title, Ayres has donecool applied work in social statistics]	<li>Meredith Broussard, <a href="http://mitpress.mit.edu/9780262038003"><cite>Artificial Unintelligence: How Computers Misunderstand the World</cite></a>	<li>Robert Cluley, <a href="https://doi.org/10.46692/9781529233391"><cite>Marketing Science Fictions: An Ethnography of Marketing Analytics, Consumer Insight, and Data Science</cite></a>	<li>Caroline Criado-Perez, <cite>Invisible Women: Data Bias in a World Designed for Men</cite>	<li>Nathan Eagle and Kate Greene, <a href="http://mitpress.mit.edu/9780262027687"><cite>Reality Mining: Using Big Data to Engineer a Better World</cite></a>	<li>Andrew Guthrie Ferguson, <cite>The Rise of Big Data Policing: Surveillance, Race and the Future of Law Enforcement</cite>	<li>Sandra Gonzalez-Bailon, <cite>Decoding the Social World: Data Science and the Unintended Consequences of Communication</cite>	<li>Timandra Harkness, <cite>Big Data: Does Size Matter?</cite>	<li>Matteo Pasquinelli and Vladan Joler, "The Nooscope manifested: AI as instrument of knowledge extractivism", <a href="https://doi.org/10.1007/s00146-020-01097-6">AI and Society</cite> <strong>36</strong> (2021): 1263--1280</a>	<li>Raj Venkatesan and Jim Lecinski, <a href="https://www.sup.org/books/title/?id=32597"><cite>The AI Marketing Canvas: A Five-Stage Road Map to Implementing Artificial Intelligence in Marketing</cite></a>	</ul> <ul>To read, historical:	<li>Jon Agar, <cite>the Government Machine: A Revolutionary History of the Computer</cite>	<li>Colin Koopman, <cite>How We Became Our Data: A Genealogy of the Informational Person</cite>	<li>Jill Lepore, <a href="https://wwnorton.com/books/9781631496103"><cite>If Then: How the Simulmatics Corporation Invented the Future</cite></a>	<li>Adrian Mackenzie, <cite>Machine Learners: Archaeology of a Data Practice</cite>	<li>Bruno J. Strasser, <cite>Collecting Experiments: Making Big Data Biology</cite>	</ul> <ul>To read, technical overviews:	<li>Bertrand S. Clarke and Jennifer L. Clarke, <cite>Predictive Statistics: Analysis and Inference Beyond Models</cite>	<li>Bertrand Clarke, Ernest Fokoue and Hao HelenZhang, <a href="http://www.springer.com/book/978-0-387-98134-5"><cite>Principles and Theory for Data Mining and Machine Learning</cite></a>	<li>Peter Flach, <a href="http://cambridge.org/9781107422223"><cite>Machine Learning: The Art and Science of Algorithms that Make Sense of Data</cite></a>	</ul> <ul>To read, topical reviews:	<li>David L. Banks and Yasmin H. Said, "Data Mining in ElectronicCommerce", <cite>Statistical Science</cite> <strong>21</strong> (2006):234--246, <a href="http://arxiv.org/abs/math.ST/0609204">math.ST/0609204</a>	<li>Colleen McCue, <cite>Data Mining and Predictive Analysis:Intelligence Gathering and Crime Analysis</cite> [To be shot after a fair trial (you should pardon the expression)]	<li>Alice Zheng, <cite>Mastering Feature Engineering: Principles & Techniques for Data Scientists</cite>	</ul> <ul>To read, technical contributions (very misc., and many now of merely-historical interest):	<li>Arvind Agarwal, Jeff M. Phillips, Suresh Venkatasubramanian, "A Unified Algorithmic Framework for Multi-Dimensional Scaling", <a href="http://arxiv.org/abs/1003.0529">arxiv:1003.0529</a>	<li>Kerstin Bunte, Michael Biehl and Barbara Hammer,"A General Framework for Dimensionality-Reducing Data Visualization Mapping",<a href="https://doi.org/10.1162/NECO_a_00250"><cite>Neural Computation</cite> <strong>24</strong> (2012): 771--804</a>	<li>Bertrand Clarke, "Desiderata for a Predictive Theory of Statistics",<a href="http://ba.stat.cmu.edu/abstracts/Clarke.php"><cite>Bayesian Analysis</cite> <strong>5</strong> (2010): 1--36</a>	<li>Graham Cormode and Ke Yi, <a href="http://cambridge.org/9781108477444"><cite>Small Summaries for Big Data</cite></a>	<li>Pavel Dmitriev and Carl Lagoze, "Mining Generalized Graph Patternsbased on UserExamples", <a href="http://arxiv.org/abs/cs.DS/0609153">cs.DS/0609153</a>	<li>Usama Fayyad, Geroges G. Grinstein and Andreas Wierse (eds.),<cite>Information Visualization in Data Mining and Knowledge Discovery</cite>	<li>Robert L. Grossman and Richard G. Larson, "State SpaceRealization Theorems for Data Mining", <a href="http://arxiv.org/abs/0901.2735">arxiv:0901.2735</a>	<li>Hillol Kargupta and Philip Chan (eds.), <a href="http://mitpress.mit.edu/0262611554"><cite>Advances in Distributed and Parallel Knolwedge Discovery</cite></a>	<li>Hillol Kargupta, Anupam Joshi, Krishnamoorthy Sivakumar and YelenaYesha, <a href="http://mitpress.mit.edu/0262612038"><cite>Data Mining: Next Generating Challenges and FutureDirections</cite></a>	<li>Nicholas M. Kiefer and C. Erik Larson, "Specification andInformational Issues in Credit Scoring", <a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=956628">SSRN/956628</a>	<li>Momin M. Malik, "A Hierarchy of Limitations in Machine Learning", <a href="http://arxiv.org/abs/2002.05193">arxiv:2002.05193</a>	<li>Giovanna Menardi, Nicola Torelli, "Training and assessing classification rules with imbalanced data",<a href="https://doi.org/10.1007/s10618-012-0295-5"><cite>Data Mining and Knowledge Discovery</cite> <strong>28</strong> (2014): 91--122</a>	<li>Michalski, Kubat, Bratko and Bratko (eds.), <cite>Machine Learningand Data Mining: Methods and Applications</cite>	<li>Rada Mihaclea and Dragomir Radev, <a href="http://cambridge.org/9780521896139"><cite>Graph-Based Natural Language Processing and Information Retrieval</cite></a>	<li>Petra Kralj Novak, Nada Lavrac and Geoffrey I. Webb,"Supervised Descriptive Rule Discovery: A Unifying Survey of Contrast Set, Emerging Pattern and Subgroup Mining", <a href="https://www.jmlr.org/papers/v10/kralj-novak09a.html"><cite>Journal of Machine Learning Research</cite> <strong>10</strong> (2009): 377--403</a>	<li>Naren Ramakrishnan and Chris Bailey-Kellogg, "Sampling Strategiesfor Mining in Data-Scarce Domains,"<a href="http://arxiv.org/abs/cs.CE/0204047">cs.CE/0204047</a>	<li>Christian H. Weiss, "Rule generation for categorical timeseries with Markov assumptions", <a href="https://doi.org/10.1007/s11222-009-9141-z"><cite>Statisticsand Computing</cite> <strong>21</strong> (2011): 1--16</a> [Variable-length Markov models]	<li>Johannes Wollbold, "Attribute Exploration of Discrete TemporalTransitions", <a href="http://arxiv.org/abs/q-bio/0701009">q-bio/0701009</a>	<li>Jun-Ming Xu, Aniruddha Bhargava, Robert Nowak, and Xiaojin Zhu, "Socioscope: Spatio-Temporal Signal Recovery from Social Media" [<a href="http://pages.cs.wisc.edu/~jerryzhu/pub/socioscope_ecml_final.pdf">PDF</a>]	<li>Mohammed Javeed Zaki, >"SPADE: An Efficient Algorithm for Mining Frequent Sequences," <cite>Machine Learning</cite> <strong>42</strong> (2001): 31--60	</ul>  </p> </div></div> <hr> <div class="footer"><P><a href="http://bactra.org/notebooks/data-mining.html">permanent link for thisnote</a> &nbsp; <a href="http://bactra.org/notebooks/data-mining.rss">RSS feed for thisnote</a></P></div><div class="foot"> <center> <cite><a href="http://bactra.org/notebooks">Notebooks</a></cite>: &nbsp; <a href="http://www.blosxom.com/">Powered by Blosxom</a> </center> </div> </body></html>