R- Programming –> Out put PDF file and a screen shot on computer with R program opened date/time of screen should be visible Write a fully executed R-Mar

R- Programming –> Out put PDF file and a screen shot on computer with R program opened date/time of screen should be visible Write a fully executed R-Markdown program and submit a pdf file solving and answering questions listed below under Problems at the end of chapter 13. For clarity, make sure to give an appropriate title to each section.
13.1 a and b.
13.2 c and d.

Mandatory:
1. Please provide me a PDF file after execution
2.  screenshot of the computer with R program opened. when executed

Please find the attached textbook and reference 13.1  and 13.2 DATA MINING
FOR BUSINESS ANALYTICS

DATA MINING
FOR BUSINESS ANALYTICS

Concepts, Techniques, and Applications in R

Galit Shmueli

Peter C. Bruce

Inbal Yahav

Nitin R. Patel

Kenneth C. Lichtendahl, Jr.

This edition first published 2018

© 2018 John Wiley & Sons, Inc.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in
any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by
law. Advice on how to obtain permission to reuse material from this title is available at
http://www.wiley.com/go/permissions.

The right of Galit Shmueli, Peter C. Bruce, Inbal Yahav, Nitin R. Patel, and Kenneth C. Lichtendahl Jr. to be
identified as the authors of this work has been asserted in accordance with law.

Registered Offices
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA

Editorial Office
111 River Street, Hoboken, NJ 07030, USA

For details of our global editorial offices, customer services, and more information about Wiley products visit us at
www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that
appears in standard print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of Warranty
The publisher and the authors make no representations or warranties with respect to the accuracy or completeness
of the contents of this work and specifically disclaim all warranties; including without limitation any implied
warranties of fitness for a particular purpose. This work is sold with the understanding that the publisher is not
engaged in rendering professional services. The advice and strategies contained herein may not be suitable for
every situation. In view of on-going research, equipment modifications, changes in governmental regulations, and
the constant flow of information relating to the use of experimental reagents, equipment, and devices, the reader
is urged to review and evaluate the information provided in the package insert or instructions for each chemical,
piece of equipment, reagent, or device for, among other things, any changes in the instructions or indication of
usage and for added warnings and precautions. The fact that an organization or website is referred to in this work
as a citation and/or potential source of further information does not mean that the author or the publisher
endorses the information the organization or website may provide or recommendations it may make. Further,
readers should be aware that websites listed in this work may have changed or disappeared between when this
works was written and when it is read. No warranty may be created or extended by any promotional statements
for this work. Neither the publisher nor the author shall be liable for any damages arising here from.

Library of Congress Cataloging-in-Publication Data applied for

Hardback: 9781118879368

Cover Design: Wiley
Cover Image: © Achim Mittler, Frankfurt am Main/Gettyimages

Set in 11.5/14.5pt BemboStd by Aptara Inc., New Delhi, India
Printed in the United States of America.

10 9 8 7 6 5 4 3 2 1

http://www.wiley.com/go/permissions

http://www.wiley.com

The beginning of wisdom is this:

Get wisdom, and whatever else you get, get insight.

– Proverbs 4:7

Contents

Foreword by Gareth James xix

Foreword by Ravi Bapna xxi

Preface to the R Edition xxiii

Acknowledgments xxvii

PART I PRELIMINARIES
CHAPTER 1 Introduction 3

1.1 What Is Business Analytics? . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 What Is Data Mining? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Data Mining and Related Terms . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Big Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Data Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.6 Why Are There So Many Different Methods? . . . . . . . . . . . . . . . . . . . 8
1.7 Terminology and Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.8 Road Maps to This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Order of Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

CHAPTER 2 Overview of the Data Mining Process 15

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Core Ideas in Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Association Rules and Recommendation Systems . . . . . . . . . . . . . . . . . 16
Predictive Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Data Reduction and Dimension Reduction . . . . . . . . . . . . . . . . . . . . 17
Data Exploration and Visualization . . . . . . . . . . . . . . . . . . . . . . . . 17
Supervised and Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . 18

2.3 The Steps in Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4 Preliminary Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Organization of Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Predicting Home Values in the West Roxbury Neighborhood . . . . . . . . . . . 21

vii

viii CONTENTS

Loading and Looking at the Data in R . . . . . . . . . . . . . . . . . . . . . . 22
Sampling from a Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Oversampling Rare Events in Classification Tasks . . . . . . . . . . . . . . . . . 25
Preprocessing and Cleaning the Data . . . . . . . . . . . . . . . . . . . . . . . 26

2.5 Predictive Power and Overfitting . . . . . . . . . . . . . . . . . . . . . . . . . 33
Overfitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Creation and Use of Data Partitions . . . . . . . . . . . . . . . . . . . . . . . 35

2.6 Building a Predictive Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Modeling Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.7 Using R for Data Mining on a Local Machine . . . . . . . . . . . . . . . . . . . 43
2.8 Automating Data Mining Solutions . . . . . . . . . . . . . . . . . . . . . . . . 43

Data Mining Software: The State of the Market (by Herb Edelstein) . . . . . . . . 45
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

PART II DATA EXPLORATION AND DIMENSION REDUCTION
CHAPTER 3 Data Visualization 55

3.1 Uses of Data Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Base R or ggplot? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

3.2 Data Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Example 1: Boston Housing Data . . . . . . . . . . . . . . . . . . . . . . . . 57
Example 2: Ridership on Amtrak Trains . . . . . . . . . . . . . . . . . . . . . . 59

3.3 Basic Charts: Bar Charts, Line Graphs, and Scatter Plots . . . . . . . . . . . . . 59
Distribution Plots: Boxplots and Histograms . . . . . . . . . . . . . . . . . . . 61
Heatmaps: Visualizing Correlations and Missing Values . . . . . . . . . . . . . . 64

3.4 Multidimensional Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Adding Variables: Color, Size, Shape, Multiple Panels, and Animation . . . . . . . 67
Manipulations: Rescaling, Aggregation and Hierarchies, Zooming, Filtering . . . . 70
Reference: Trend Lines and Labels . . . . . . . . . . . . . . . . . . . . . . . . 74
Scaling up to Large Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Multivariate Plot: Parallel Coordinates Plot . . . . . . . . . . . . . . . . . . . . 75
Interactive Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

3.5 Specialized Visualizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Visualizing Networked Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Visualizing Hierarchical Data: Treemaps . . . . . . . . . . . . . . . . . . . . . 82
Visualizing Geographical Data: Map Charts . . . . . . . . . . . . . . . . . . . . 83

3.6 Summary: Major Visualizations and Operations, by Data Mining Goal . . . . . . . 86
Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
Time Series Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

CHAPTER 4 Dimension Reduction 91

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.2 Curse of Dimensionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

CONTENTS ix

4.3 Practical Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Example 1: House Prices in Boston . . . . . . . . . . . . . . . . . . . . . . . 93

4.4 Data Summaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Summary Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Aggregation and Pivot Tables . . . . . . . . . . . . . . . . . . . . . . . . . . 96

4.5 Correlation Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

4.6 Reducing the Number of Categories in Categorical Variables . . . . . . . . . . . 99

4.7 Converting a Categorical Variable to a Numerical Variable . . . . . . . . . . . . 99

4.8 Principal Components Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Example 2: Breakfast Cereals . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Principal Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Normalizing the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Using Principal Components for Classification and Prediction . . . . . . . . . . . 109

4.9 Dimension Reduction Using Regression Models . . . . . . . . . . . . . . . . . . 111

4.10 Dimension Reduction Using Classification and Regression Trees . . . . . . . . . . 111

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

PART III PERFORMANCE EVALUATION

CHAPTER 5 Evaluating Predictive Performance 117

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

5.2 Evaluating Predictive Performance . . . . . . . . . . . . . . . . . . . . . . . . 118

Naive Benchmark: The Average . . . . . . . . . . . . . . . . . . . . . . . . . 118

Prediction Accuracy Measures . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Comparing Training and Validation Performance . . . . . . . . . . . . . . . . . 121

Lift Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

5.3 Judging Classifier Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Benchmark: The Naive Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Class Separation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

The Confusion (Classification) Matrix . . . . . . . . . . . . . . . . . . . . . . . 124

Using the Validation Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Accuracy Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Propensities and Cutoff for Classification . . . . . . . . . . . . . . . . . . . . . 127

Performance in Case of Unequal Importance of Classes . . . . . . . . . . . . . . 131

Asymmetric Misclassification Costs . . . . . . . . . . . . . . . . . . . . . . . . 133

Generalization to More Than Two Classes . . . . . . . . . . . . . . . . . . . . . 135

5.4 Judging Ranking Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 136

Lift Charts for Binary Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

Decile Lift Charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

Beyond Two Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Lift Charts Incorporating Costs and Benefits . . . . . . . . . . . . . . . . . . . 139

Lift as a Function of Cutoff . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

5.5 Oversampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

Oversampling the Training Set . . . . . . . . . . . . . . . . . . . . . . . . . . 144

x CONTENTS

Evaluating Model Performance Using a Non-oversampled Validation Set . . . . . . 144
Evaluating Model Performance if Only Oversampled Validation Set Exists . . . . . 144

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

PART IV PREDICTION AND CLASSIFICATION METHODS
CHAPTER 6 Multiple Linear Regression 153

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
6.2 Explanatory vs. Predictive Modeling . . . . . . . . . . . . . . . . . . . . . . . 154
6.3 Estimating the Regression Equation and Prediction . . . . . . . . . . . . . . . . 156

Example: Predicting the Price of Used Toyota Corolla Cars . . . . . . . . . . . . 156
6.4 Variable Selection in Linear Regression . . . . . . . . . . . . . . . . . . . . . 161

Reducing the Number of Predictors . . . . . . . . . . . . . . . . . . . . . . . 161
How to Reduce the Number of Predictors . . . . . . . . . . . . . . . . . . . . . 162

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

CHAPTER 7 k-Nearest Neighbors (kNN) 173

7.1 The k-NN Classifier (Categorical Outcome) . . . . . . . . . . . . . . . . . . . . 173
Determining Neighbors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Classification Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Example: Riding Mowers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
Choosing k . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
Setting the Cutoff Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
k-NN with More Than Two Classes . . . . . . . . . . . . . . . . . . . . . . . . 180
Converting Categorical Variables to Binary Dummies . . . . . . . . . . . . . . . 180

7.2 k-NN for a Numerical Outcome . . . . . . . . . . . . . . . . . . . . . . . . . . 180
7.3 Advantages and Shortcomings of k-NN Algorithms . . . . . . . . . . . . . . . . 182
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

CHAPTER 8 The Naive Bayes Classifier 187

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
Cutoff Probability Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
Example 1: Predicting Fraudulent Financial Reporting . . . . . . . . . . . . . . 188

8.2 Applying the Full (Exact) Bayesian Classifier . . . . . . . . . . . . . . . . . . . 189
Using the “Assign to the Most Probable Class” Method . . . . . . . . . . . . . . 190
Using the Cutoff Probability Method . . . . . . . . . . . . . . . . . . . . . . . 190
Practical Difficulty with the Complete (Exact) Bayes Procedure . . . . . . . . . . 190
Solution: Naive Bayes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
The Naive Bayes Assumption of Conditional Independence . . . . . . . . . . . . 192
Using the Cutoff Probability Method . . . . . . . . . . . . . . . . . . . . . . . 192
Example 2: Predicting Fraudulent Financial Reports, Two Predictors . . . . . . . 193
Example 3: Predicting Delayed Flights . . . . . . . . . . . . . . . . . . . . . . 194

8.3 Advantages and Shortcomings of the Naive Bayes Classifier . . . . . . . . . . . 199
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

CONTENTS xi

CHAPTER 9 Classification and Regression Trees 205

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

9.2 Classification Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

Recursive Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

Example 1: Riding Mowers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

Measures of Impurity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

Tree Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Classifying a New Record . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

9.3 Evaluating the Performance of a Classification Tree . . . . . . . . . . . . . . . . 215

Example 2: Acceptance of Personal Loan . . . . . . . . . . . . . . . . . . . . . 215

9.4 Avoiding Overfitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

Stopping Tree Growth: Conditional Inference Trees . . . . . . . . . . . . . . . . 221

Pruning the Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Cross-Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Best-Pruned Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

9.5 Classification Rules from Trees . . . . . . . . . . . . . . . . . . . . . . . . . . 226

9.6 Classification Trees for More Than Two Classes . . . . . . . . . . . . . . . . . . 227

9.7 Regression Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

Measuring Impurity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

Evaluating Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

9.8 Improving Prediction: Random Forests and Boosted Trees . . . . . . . . . . . . 229

Random Forests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

Boosted Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

9.9 Advantages and Weaknesses of a Tree . . . . . . . . . . . . . . . . . . . . . . 232

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

CHAPTER 10 Logistic Regression 237

10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

10.2 The Logistic Regression Model . . . . . . . . . . . . . . . . . . . . . . . . . . 239

10.3 Example: Acceptance of Personal Loan . . . . . . . . . . . . . . . . . . . . . . 240

Model with a Single Predictor . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Estimating the Logistic Model from Data: Computing Parameter Estimates . . . . 243

Interpreting Results in Terms of Odds (for a Profiling Goal) . . . . . . . . . . . . 244

10.4 Evaluating Classification Performance . . . . . . . . . . . . . . . . . . . . . . 247

Variable Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

10.5 Example of Complete Analysis: Predicting Delayed Flights . . . . . . . . . . . . 250

Data Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Model-Fitting and Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 254

Model Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

Model Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

Variable Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

10.6 Appendix: Logistic Regression for Profiling . . . . . . . . . . . . . . . . . . . . 259

Appendix A: Why Linear Regression Is Problematic for a Categorical Outcome . . . 259

xii CONTENTS

Appendix B: Evaluating Explanatory Power . . . . . . . . . . . . . . . . . . . . 261
Appendix C: Logistic Regression for More Than Two Classes . . . . . . . . . . . . 264

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

CHAPTER 11 Neural Nets 271

11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
11.2 Concept and Structure of a Neural Network . . . . . . . . . . . . . . . . . . . . 272
11.3 Fitting a Network to Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

Example 1: Tiny Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
Computing Output of Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Preprocessing the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
Training the Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
Example 2: Classifying Accident Severity . . . . . . . . . . . . . . . . . . . . . 282
Avoiding Overfitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Using the Output for Prediction and Classification . . . . . . . . . . . . . . . . 283

11.4 Required User Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
11.5 Exploring the Relationship Between Predictors and Outcome . . . . . . . . . . . 287
11.6 Advantages and Weaknesses of Neural Networks . . . . . . . . . . . . . . . . . 288
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

CHAPTER 12 Discriminant Analysis 293

12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
Example 1: Riding Mowers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
Example 2: Personal Loan Acceptance . . . . . . . . . . . . . . . . . . . . . . 294

12.2 Distance of a Record from a Class . . . . . . . . . . . . . . . . . . . . . . . . 296
12.3 Fisher’s Linear Classification Functions . . . . . . . . . . . . . . . . . . . . . . 297
12.4 Classification Performance of Discriminant Analysis . . . . . . . . . . . . . . . 300
12.5 Prior Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
12.6 Unequal Misclassification Costs . . . . . . . . . . . . . . . . . . . . . . . . . 302
12.7 Classifying More Than Two Classes . . . . . . . . . . . . . . . . . . . . . . . . 303

Example 3: Medical Dispatch to Accident Scenes . . . . . . . . . . . . . . . . . 303
12.8 Advantages and Weaknesses . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

CHAPTER 13 Combining Methods: Ensembles and Uplift Modeling 311

13.1 Ensembles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
Why Ensembles Can Improve Predictive Power . . . . . . . . . . . . . . . . . . 312
Simple Averaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
Bagging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Boosting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Bagging and Boosting in R . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Advantages and Weaknesses of Ensembles . . . . . . . . . . . . . . . . . . . . 315

13.2 Uplift (Persuasion) Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
A-B Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

CONTENTS xiii

Uplift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
Gathering the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
A Simple Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
Modeling Individual Uplift . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
Computing Uplift with R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
Using the Results of an Uplift Model . . . . . . . . . . . . . . . . . . . . . . . 322

13.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325

PART V MINING RELATIONSHIPS AMONG RECORDS
CHAPTER 14 Association Rules and Collaborative Filtering 329

14.1 Association Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
Discovering Association Rules in Transaction Databases . . . . . . . . . . . . . 330
Example 1: Synthetic Data on Purchases of Phone Faceplates . . . . . . . . . . 330
Generating Candidate Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
The Apriori Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
Selecting Strong Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
Data Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
The Process of Rule Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 336
Interpreting the Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
Rules and Chance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
Example 2: Rules for Similar Book Purchases . . . . . . . . . . . . . . . . . . . 340

14.2 Collaborative Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
Data Type and Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
Example 3: Netflix Prize Contest . . . . . . . . . . . . . . . . . . . . . . . . . 343
User-Based Collaborative Filtering: “People Like You” . . . . . . . . . . . . . . 344
Item-Based Collaborative Filtering . . . . . . . . . . . . . . . . . . . . . . . . 347
Advantages and Weaknesses of Collaborative Filtering . . . . . . . . . . . . . . 348
Collaborative Filtering vs. Association Rules . . . . . . . . . . . . . . . . . . . 349

14.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352

CHAPTER 15 Cluster Analysis 357

15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
Example: Public Utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359

15.2 Measuring Distance Between Two Records . . . . . . . . . . . . . . . . . . . . 361
Euclidean Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
Normalizing Numerical Measurements . . . . . . . . . . . . . . . . . . . . . . 362
Other Distance Measures for Numerical Data . . . . . . . . . . . . . . . . . . . 362
Distance Measures for Categorical Data . . . . . . . . . . . . . . . . . . . . . . 365
Distance Measures for Mixed Data . . . . . . . . . . . . . . . . . . . . . . . . 366

15.3 Measuring Distance Between Two Clusters . . . . . . . . . . . . . . . . . . . . 366
Minimum Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
Maximum Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366

xiv CONTENTS

Average Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

Centroid Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

15.4 Hierarchical (Agglomerative) Clustering . . . . . . . . . . . . . . . . . . . . . 368

Single Linkage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369

Complete Linkage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

Average Linkage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

Centroid Linkage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

Ward’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

Dendrograms: Displaying Clustering Process and Results . . . . . . . . . . . . . 371

Validating Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373

Limitations of Hierarchical Clustering . . . . . . . . . . . . . . . . . . . . . . 375

15.5 Non-Hierarchical Clustering: The k-Means Algorithm . . . . . . . . . . . . . . . 376

Choosing the Number of Clusters (k) . . . . . . . . . . . . . . . . . . . . . . . 377

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382

PART VI FORECASTING TIME SERIES

CHAPTER 16 Handling Time Series 387

16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387

16.2 Descriptive vs. Predictive Modeling . . . . . . . . . . . . . . . . . . . . . . . 389

16.3 Popular Forecasting Methods in Business . . . . . . . . . . . . . . . . . . . . . 389

Combining Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389

16.4 Time Series Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390

Example: Ridership on Amtrak Trains . . . . . . . . . . . . . . . . . . . . . . . 390

16.5 Data-Partitioning and Performance Evaluation . . . . . . . . . . . . . . . . . . 395

Benchmark Performance: Naive Forecasts . . . . . . . . . . . . . . . . . . . . 395

Generating Future Forecasts . . . . . . . . . . . . . . . . . . . . . . . . . . . 396

Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398

CHAPTER 17 Regression-Based Forecasting 401

17.1 A Model with Trend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401

Linear Trend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401

Exponential Trend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

Polynomial Trend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407

17.2 A Model with Seasonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407

17.3 A Model with Trend and Seasonality . . . . . . . . . . . . . . . . . . . . . . . 411

17.4 Autocorrelation and ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . 412

Computing Autocorrelation . . . . . . . . . . . . . . . . . …