In this chapter, we will cover the topics of tidying your data in preparation for predictive modeling, performance metrics, cross-validation, and learning curves.
In statistics, it is an accepted concept that there are two types of data, which are:
- Untidy
- Tidy
Untidy data is considered to be raw or messy; tidy data is data that has gone through a quality assurance process and is ready to be used.
Before we get started with discussing the process of tidying data, it would be very prudent to point out that whatever you do to tidy your data, you should be sure to:
- Create and save your scripts so that you can use them again for new or similar data sources. This is referred to as reusability. Why spend time recreating the same code, rules, or logic if you don't have to? This applies to new data within the same project (that the scripts were developed for) or new projects you may be involved with in the future.
- Tidy your data as "far upstream" as possible, perhaps even at the original source. In other words, save and maintain the original data, but use programmatic scripts to clean it, fix mistakes, and save that cleaned dataset for further analysis.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.