Chapter 2. Preparing for Analysis – Data Cleansing and Manipulation

In this chapter, we will cover the following topics:

  • Getting a sense of your data structure with R
  • Preparing your data for analysis with the tidyr package
  • Detecting missing values
  • Substituting missing values by interpolation
  • Detecting and removing outliers
  • Performing data filtering activities

Introduction

Some studies estimate that data preparation activities account for 80 percent of the time invested in data science projects.

I know you will not be surprised reading this number. Data preparation is the phase in data science projects where you take your data from the chaotic world around you and fit it into some precise structures and standards.

This is absolutely not a simple task and involves a great number of techniques that basically let you change the structure of your data and ensure you can work with it.

This chapter will show you recipes that should give you the ability to prepare the data you got from the previous chapter, no matter how it was structured when you acquired it in R.

We will look at the two main activities performed during the data preparation phase:

  • Data cleansing: This involves identification and treatment of outliers and missing values
  • Data manipulation: Here, the main aim is to make the data structure fit some specific rule, which will let the user employ it for analysis
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset