Challenge 3 Solutions

Code

library(tidyverse)

knitr::opts_chunk$set(echo = TRUE, warning=FALSE, message=FALSE)

Challenge Overview

Today’s challenge is to:

1. read in a data set, and describe the data set using both words and any supporting information (e.g., tables, etc)
2. identify what needs to be done to tidy the current data
3. anticipate the shape of pivoted data
4. pivot the data into tidy format using pivot_longer

Read in data

Read in one (or more) of the following datasets, using the correct R package and command.

• animal_weights.csv ⭐
• eggs_tidy.csv ⭐⭐ or organiceggpoultry.xls ⭐⭐⭐
• australian_marriage*.xls ⭐⭐⭐
• USA Households*.xlsx ⭐⭐⭐⭐
• sce_labor_chart_data_public.xlsx 🌟🌟🌟🌟🌟

Code

# read in data (I am having trouble setting my working directory):

animal_weight <- read_csv("_data/animal_weight.csv")
# This data set, which appears to represent animal weights by IPCC area. The IPCC, or Intergovernmental Panel on Climate Change, is the United Nations body for assessing the science related to climate change, so this is clearly data collected from a governmental source. After reading in the data set, we can see that the data set has 9 rows/cases and 17 columns/variables. One variable represents IPCC areas, and the remaining 16 variables represent varying species/breeds of livestock. The values for each observation represent the animals' weight.

Briefly describe the data

Describe the data, and be sure to comment on why you are planning to pivot it to make it “tidy”

Anticipate the End Result

The first step in pivoting the data is to try to come up with a concrete vision of what the end product should look like - that way you will know whether or not your pivoting was successful.

One easy way to do this is to think about the dimensions of your current data (tibble, dataframe, or matrix), and then calculate what the dimensions of the pivoted data should be.

Suppose you have a dataset with \(n\) rows and \(k\) variables. In our example, 3 of the variables are used to identify a case, so you will be pivoting \(k-3\) variables into a longer format where the \(k-3\) variable names will move into the names_to variable and the current values in each of those columns will move into the values_to variable. Therefore, we would expect \(n * (k-3)\) rows in the pivoted dataframe!

Example: find current and future data dimensions

Lets see if this works with a simple example.

Code

df<-tibble(country = rep(c("Mexico", "USA", "France"),2),
           year = rep(c(1980,1990), 3), 
           trade = rep(c("NAFTA", "NAFTA", "EU"),2),
           outgoing = rnorm(6, mean=1000, sd=500),
           incoming = rlogis(6, location=1000, 
                             scale = 400))
df

# A tibble: 6 × 5
  country  year trade outgoing incoming
  <chr>   <dbl> <chr>    <dbl>    <dbl>
1 Mexico   1980 NAFTA    1351.     827.
2 USA      1990 NAFTA     682.     811.
3 France   1980 EU       1432.    1272.
4 Mexico   1990 NAFTA     881.    1100.
5 USA      1980 NAFTA    1213.    1460.
6 France   1990 EU        460.     170.

Code

#existing rows/cases
nrow(df)

[1] 6

Code

#existing columns/cases
ncol(df)

[1] 5

Code

#expected rows/cases
nrow(df) * (ncol(df)-3)

[1] 12

Code

# expected columns 
3 + 2

[1] 5

Or simple example has \(n = 6\) rows and \(k - 3 = 2\) variables being pivoted, so we expect a new dataframe to have \(n * 2 = 12\) rows x \(3 + 2 = 5\) columns.

Challenge: Describe the final dimensions

Document your work here.

Because each row represents several observations (16 observations about the weights of varying species/breeds of livestock for every case), we will need to tidy the data. To do this, I will consolidate the livestock species/breeds into one category, then create a separate column/variable that captures weight by case.

We know that there are 9 cases and 16 livestock-specific variables in this data set, so the final pivoted data table should have 3 columns (IPCC area, livestock, weight) and 144 rows.

Code

# calculation for number of rows:
9*16

[1] 144

Any additional comments?

Pivot the Data

Now we will pivot the data, and compare our pivoted data dimensions to the dimensions calculated above as a “sanity” check.

Example

Code

df<-pivot_longer(df, col = c(outgoing, incoming),
                 names_to="trade_direction",
                 values_to = "trade_value")
df

# A tibble: 12 × 5
   country  year trade trade_direction trade_value
   <chr>   <dbl> <chr> <chr>                 <dbl>
 1 Mexico   1980 NAFTA outgoing              1351.
 2 Mexico   1980 NAFTA incoming               827.
 3 USA      1990 NAFTA outgoing               682.
 4 USA      1990 NAFTA incoming               811.
 5 France   1980 EU    outgoing              1432.
 6 France   1980 EU    incoming              1272.
 7 Mexico   1990 NAFTA outgoing               881.
 8 Mexico   1990 NAFTA incoming              1100.
 9 USA      1980 NAFTA outgoing              1213.
10 USA      1980 NAFTA incoming              1460.
11 France   1990 EU    outgoing               460.
12 France   1990 EU    incoming               170.

Yes, once it is pivoted long, our resulting data are \(12x5\) - exactly what we expected!

Challenge: Pivot the Chosen Data

Document your work here. What will a new “case” be once you have pivoted the data? How does it meet requirements for tidy data?

Code

# pivot data

animals_pivot <- pivot_longer(animals,col=-'IPCC Area', names_to="Livestock", values_to="Weight")

Error in pivot_longer(animals, col = -"IPCC Area", names_to = "Livestock", : object 'animals' not found

Now that I have pivoted this data, I have a case that includes Indian Subcontinent (IPCC Area), Cattle - Dairy (Livestock), and 275 (Weight). This meets tidy data requirements because each variable has its own column, each observation has its own row, and each value has its own cell.

Any additional comments?