Challenge 3

challenge_3

Tidy Data: Pivoting

Author

Daniel Manning

Published

January 2, 2023

Code

library(tidyverse)
knitr::opts_chunk$set(echo = TRUE, warning=FALSE, message=FALSE)

Challenge Overview

Today’s challenge is to:

read in a data set, and describe the data set using both words and any supporting information (e.g., tables, etc)
identify what needs to be done to tidy the current data
anticipate the shape of pivoted data
pivot the data into tidy format using pivot_longer

Read in data

Read in one (or more) of the following datasets, using the correct R package and command.

animal_weights.csv ⭐
eggs_tidy.csv ⭐⭐ or organicpoultry.xls ⭐⭐⭐
australian_marriage*.xlsx ⭐⭐⭐
USA Households*.xlsx ⭐⭐⭐⭐
sce_labor_chart_data_public.csv 🌟🌟🌟🌟🌟

Code

weight <- here("posts","_data","animal_weight.csv")%>%
  read_csv()

Error in here("posts", "_data", "animal_weight.csv"): could not find function "here"

Code

head(weight)

Error in head(weight): object 'weight' not found

Code

weight

Error in eval(expr, envir, enclos): object 'weight' not found

Briefly describe the data

Describe the data, and be sure to comment on why you are planning to pivot it to make it “tidy”

This data describes values for different animals (with specific subtypes) in 9 difference IPCC Areas. I’m planning to collapse the 16 animal cases into one variable called Animal so there is one Value column.

Anticipate the End Result

The first step in pivoting the data is to try to come up with a concrete vision of what the end product should look like - that way you will know whether or not your pivoting was successful.

One easy way to do this is to think about the dimensions of your current data (tibble, dataframe, or matrix), and then calculate what the dimensions of the pivoted data should be.

Suppose you have a dataset with \(n\) rows and \(k\) variables. In our example, 3 of the variables are used to identify a case, so you will be pivoting \(k-3\) variables into a longer format where the \(k-3\) variable names will move into the names_to variable and the current values in each of those columns will move into the values_to variable. Therefore, we would expect \(n * (k-3)\) rows in the pivoted dataframe!

Example: find current and future data dimensions

Lets see if this works with a simple example.

Code

df<-tibble(country = rep(c("Mexico", "USA", "France"),2),
           year = rep(c(1980,1990), 3), 
           trade = rep(c("NAFTA", "NAFTA", "EU"),2),
           outgoing = rnorm(6, mean=1000, sd=500),
           incoming = rlogis(6, location=1000, 
                             scale = 400))
df

# A tibble: 6 × 5
  country  year trade outgoing incoming
  <chr>   <dbl> <chr>    <dbl>    <dbl>
1 Mexico   1980 NAFTA    1310.    -451.
2 USA      1990 NAFTA     762.     231.
3 France   1980 EU       1249.    1389.
4 Mexico   1990 NAFTA    1113.     601.
5 USA      1980 NAFTA     248.   -1744.
6 France   1990 EU        657.     866.

Code

#existing rows/cases
nrow(df)

[1] 6

Code

#existing columns/cases
ncol(df)

[1] 5

Code

#expected rows/cases
nrow(df) * (ncol(df)-3)

[1] 12

Code

# expected columns

Or simple example has \(n = 6\) rows and \(k - 3 = 2\) variables being pivoted, so we expect a new dataframe to have \(n * 2 = 12\) rows x \(3 + 2 = 5\) columns.

Challenge: Describe the final dimensions

Document your work here.

Code

#existing rows/cases
nrow(weight)

Error in nrow(weight): object 'weight' not found

Code

#existing columns/cases
ncol(weight)

Error in ncol(weight): object 'weight' not found

Code

#expected rows/cases
nrow(weight) * 16

Error in nrow(weight): object 'weight' not found

Code

# expected columns 
1 + 2

[1] 3

Any additional comments?

Pivot the Data

Now we will pivot the data, and compare our pivoted data dimensions to the dimensions calculated above as a “sanity” check.

Example

Code

df<-pivot_longer(df, col = c(),
                 names_to="trade_direction",
                 values_to = "trade_value")

Error in `pivot_longer()`:
! `cols` must select at least one column.

Code

df

# A tibble: 6 × 5
  country  year trade outgoing incoming
  <chr>   <dbl> <chr>    <dbl>    <dbl>
1 Mexico   1980 NAFTA    1310.    -451.
2 USA      1990 NAFTA     762.     231.
3 France   1980 EU       1249.    1389.
4 Mexico   1990 NAFTA    1113.     601.
5 USA      1980 NAFTA     248.   -1744.
6 France   1990 EU        657.     866.

Yes, once it is pivoted long, our resulting data are \(12x5\) - exactly what we expected!

Challenge: Pivot the Chosen Data

Document your work here. What will a new “case” be once you have pivoted the data? How does it meet requirements for tidy data?

The new variable is Animal with cases: ‘Cattle - dairy’, ‘Cattle - non-dairy’, ‘Buffaloes’, ‘Swine - market’, ‘Swine - breeding’, ‘Chicken - Broilers’, ‘Chicken - Layers’, ‘Ducks’, ‘Turkeys’, ‘Sheep’, ‘Goats’, ‘Horses’, ‘Asses’, ‘Mules’, ‘Camels’, ‘Llamas’. It meets the requirements for tidy data because now each column is a variable and each row is an occurence of that variable.

Code

weight_tidy<-pivot_longer(weight, col = c('Cattle - dairy','Cattle - non-dairy','Buffaloes','Swine - market','Swine - breeding','Chicken - Broilers','Chicken - Layers','Ducks','Turkeys','Sheep','Goats','Horses','Asses','Mules','Camels','Llamas'),
                 names_to="Animal",
                 values_to = "Value")

Error in pivot_longer(weight, col = c("Cattle - dairy", "Cattle - non-dairy", : object 'weight' not found

Code

weight_tidy

Error in eval(expr, envir, enclos): object 'weight_tidy' not found

Any additional comments?