Project 2

project 2

projects

Exploring temperature and rainfall in Australia

Author

Affiliations

Johns Hopkins Bloomberg School of Public Health Department of Biostatistics

This project, as the rest of the course, is adapted from the version Stephanie C. Hicks designed and maintained in 2021 and 2022. Check the recent changes to this file through the GitHub history.

Background

Due date: September 29th at 11:59pm

The goal of this assignment is to practice designing and writing functions along with practicing our tidyverse skills that we learned in our previous project. Writing functions involves thinking about how code should be divided up and what the interface/arguments should be. In addition, you need to think about what the function will return as output.

To submit your project

Please write up your project using R Markdown and processed with knitr. Compile your document as an HTML file and submit your HTML file to the dropbox on Courseplus. Please show all your code (i.e. make sure to set echo = TRUE) for each of the answers to each part.

Install packages

Before attempting this assignment, you should first install the following packages, if they are not already installed:

install.packages("tidyverse")
install.packages("tidytuesdayR")

Part 1: Fun with functions

In this part, we are going to practice creating functions.

Part 1A: Exponential transformation

The exponential of a number can be written as an infinite series expansion of the form \[ \exp(x) = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \cdots \] Of course, we cannot compute an infinite series by the end of this term and so we must truncate it at a certain point in the series. The truncated sum of terms represents an approximation to the true exponential, but the approximation may be usable.

Write a function that computes the exponential of a number using the truncated series expansion. The function should take two arguments:

x: the number to be exponentiated
k: the number of terms to be used in the series expansion beyond the constant 1. The value of k is always \(\geq 1\).

For example, if \(k = 1\), then the Exp function should return the number \(1 + x\). If \(k = 2\), then you should return the number \(1 + x + x^2/2!\).

Include at least one example of output using your function.

Note

You can assume that the input value x will always be a single number.
You can assume that the value k will always be an integer \(\geq 1\).
Do not use the exp() function in R.
The factorial() function can be used to compute factorials.

Exp <- function(x, k) {
    # Add your solution here
}

Part 1B: Sample mean and sample standard deviation

Next, write two functions called sample_mean() and sample_sd() that takes as input a vector of data of length \(N\) and calculates the sample average and sample standard deviation for the set of \(N\) observations.

\[ \bar{x} = \frac{1}{N} \sum_{i=1}^n x_i \] \[ s = \sqrt{\frac{1}{N-1} \sum_{i=1}^N (x_i - \overline{x})^2} \] As a primer, the \(\sum_{i = 1}^3 x_i\) is equivalent to doing the sum of \(x_i\) starting from \(i = 1\), increasing \(i\) by 1 each then, up to when \(i = 3\). That is, \(x_1 + x_2 + x_3\).

Include at least one example of output using your functions.

Note

You can assume that the input value x will always be a vector of numbers of length N.
Do not use the mean() and sd() functions in R.

sample_mean <- function(x) {
    # Add your solution here
}

sample_sd <- function(x) {
    # Add your solution here
}

Part 1C: Confidence intervals

Next, write a function called calculate_CI() that:

There should be two inputs to the calculate_CI(). First, it should take as input a vector of data of length \(N\). Second, the function should also have a conf (\(=1-\alpha\)) argument that allows the confidence interval to be adapted for different \(\alpha\).
Calculates a confidence interval (CI) (e.g. a 95% CI) for the estimate of the mean in the population. If you are not familiar with confidence intervals, it is an interval that contains the population parameter with probability \(1-\alpha\) taking on this form

\[ \bar{x} \pm t_{\alpha/2, N-1} s_{\bar{x}} = ( \bar{x} - t_{\alpha/2, N-1} s_{\bar{x}}, \bar{x} + t_{\alpha/2, N-1} s_{\bar{x}} ) \]

where \(t_{\alpha/2, N-1}\) is the value needed to generate an area of \(\alpha / 2\) in each tail of the \(t\)-distribution with \(N-1\) degrees of freedom and \(s_{\bar{x}} = \frac{s}{\sqrt{N}}\) is the standard error of the mean. For example, if we pick a 95% confidence interval and \(N\)=50, then you can calculate \(t_{\alpha/2, N-1}\) as

alpha <- 1 - 0.95
degrees_freedom <- 50 - 1
t_score <- qt(p = alpha / 2, df = degrees_freedom, lower.tail = FALSE)

Returns a named vector of length 2, where the first value is the lower_bound, the second value is the upper_bound.

calculate_CI <- function(x, conf = 0.95) {
    # Add your solution here
}

Include example of output from your function showing the output when using two different levels of conf.

Note

If you want to check if your function output matches an existing function in R, consider a vector \(x\) of length \(N\) and see if the following two code chunks match.

calculate_CI(x, conf = 0.95)

dat <- data.frame(x = x)
fit <- lm(x ~ 1, dat)

# Calculate a 95% confidence interval
confint(fit, level = 0.95)

Part 2: Wrangling data

In this part, we will practice our wrangling skills with the tidyverse that we learned about in module 1.

Data

The two datasets for this part of the assignment comes from TidyTuesday. Specifically, we will use the following data from January 2020, which I have provided for you below:

tuesdata <- tidytuesdayR::tt_load("2020-01-07")
rainfall <- tuesdata$rainfall
temperature <- tuesdata$temperature

However, to avoid re-downloading data, we will check to see if those files already exist using an if() statement:

library(here)
if (!file.exists(here("data", "tuesdata_rainfall.RDS"))) {
    tuesdata <- tidytuesdayR::tt_load("2020-01-07")
    rainfall <- tuesdata$rainfall
    temperature <- tuesdata$temperature

    # save the files to RDS objects
    saveRDS(tuesdata$rainfall, file = here("data", "tuesdata_rainfall.RDS"))
    saveRDS(tuesdata$temperature, file = here("data", "tuesdata_temperature.RDS"))
}

Note

The above code will only run if it cannot find the path to the tuesdata_rainfall.RDS on your computer. Then, we can just read in these files every time we knit the R Markdown, instead of re-downloading them every time.

Let’s load the datasets

rainfall <- readRDS(here("data", "tuesdata_rainfall.RDS"))
temperature <- readRDS(here("data", "tuesdata_temperature.RDS"))

Now we can look at the data with glimpse()

library(tidyverse)

glimpse(rainfall)

Rows: 179,273
Columns: 11
$ station_code <chr> "009151", "009151", "009151", "009151", "009151", "009151…
$ city_name    <chr> "Perth", "Perth", "Perth", "Perth", "Perth", "Perth", "Pe…
$ year         <dbl> 1967, 1967, 1967, 1967, 1967, 1967, 1967, 1967, 1967, 196…
$ month        <chr> "01", "01", "01", "01", "01", "01", "01", "01", "01", "01…
$ day          <chr> "01", "02", "03", "04", "05", "06", "07", "08", "09", "10…
$ rainfall     <dbl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, N…
$ period       <dbl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, N…
$ quality      <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, N…
$ lat          <dbl> -31.96, -31.96, -31.96, -31.96, -31.96, -31.96, -31.96, -…
$ long         <dbl> 115.79, 115.79, 115.79, 115.79, 115.79, 115.79, 115.79, 1…
$ station_name <chr> "Subiaco Wastewater Treatment Plant", "Subiaco Wastewater…

glimpse(temperature)

Rows: 528,278
Columns: 5
$ city_name   <chr> "PERTH", "PERTH", "PERTH", "PERTH", "PERTH", "PERTH", "PER…
$ date        <date> 1910-01-01, 1910-01-02, 1910-01-03, 1910-01-04, 1910-01-0…
$ temperature <dbl> 26.7, 27.0, 27.5, 24.0, 24.8, 24.4, 25.3, 28.0, 32.6, 35.9…
$ temp_type   <chr> "max", "max", "max", "max", "max", "max", "max", "max", "m…
$ site_name   <chr> "PERTH AIRPORT", "PERTH AIRPORT", "PERTH AIRPORT", "PERTH …

If we look at the TidyTuesday github repo from 2020, we see this dataset contains temperature and rainfall data from Australia.

Here is a data dictionary for what all the column names mean:

https://github.com/rfordatascience/tidytuesday/blob/master/data/2020/2020-01-07/readme.md#data-dictionary

Tasks

Using the rainfall and temperature data, perform the following steps and create a new data frame called df:

Start with rainfall dataset and drop any rows with NAs.
Create a new column titled date that combines the columns year, month, day into one column separated by “-”. (e.g. “2020-01-01”). This column should not be a character, but should be recognized as a date. (Hint: check out the ymd() function in lubridate R package). You will also want to add a column that just keeps the year.
Using the city_name column, convert the city names (character strings) to all upper case.
Join this wrangled rainfall dataset with the temperature dataset such that it includes only observations that are in both data frames. (Hint: there are two keys that you will need to join the two datasets together). (Hint: If all has gone well thus far, you should have a dataset with 83,964 rows and 13 columns).

Note

You may need to use functions outside these packages to obtain this result, in particular you may find the functions drop_na() from tidyr and str_to_upper() function from stringr useful.

# Add your solution here

Part 3: Data visualization

In this part, we will practice our ggplot2 plotting skills within the tidyverse starting with our wrangled df data from Part 2. For full credit in this part (and for all plots that you make), your plots should include:

An overall title for the plot and a subtitle summarizing key trends that you found. Also include a caption in the figure.
There should be an informative x-axis and y-axis label.

Consider playing around with the theme() function to make the figure shine, including playing with background colors, font, etc.

Part 3A: Plotting temperature data over time

Use the functions in ggplot2 package to make a line plot of the max and min temperature (y-axis) over time (x-axis) for each city in our wrangled data from Part 2. You should only consider years 2014 and onwards. For full credit, your plot should include:

For a given city, the min and max temperature should both appear on the plot, but they should be two different colors.
Use a facet function to facet by city_name to show all cities in one figure.

# Add your solution here

Part 3B: Plotting rainfall over time

Here we want to explore the distribution of rainfall (log scale) with histograms for a given city (indicated by the city_name column) for a given year (indicated by the year column) so we can make some exploratory plots of the data.

Note

You are again using the wrangled data from Part 2.

The following code plots the data from one city (city_name == "PERTH") in a given year (year == 2000).

df %>%
    filter(city_name == "PERTH", year == 2000) %>%
    ggplot(aes(log(rainfall))) +
    geom_histogram()

While this code is useful, it only provides us information on one city in one year. We could cut and paste this code to look at other cities/years, but that can be error prone and just plain messy.

The aim here is to design and implement a function that can be re-used to visualize all of the data in this dataset.

There are 2 aspects that may vary in the dataset: The city_name and the year. Note that not all combinations of city_name and year have measurements.
Your function should take as input two arguments city_name and year.
Given the input from the user, your function should return a single histogram for that input. Furthermore, the data should be readable on that plot so that it is in fact useful. It should be possible visualize the entire dataset with your function (through repeated calls to your function).
If the user enters an input that does not exist in the dataset, your function should catch that and report an error (via the stop() function).

For this section,

Write a short description of how you chose to design your function and why.
Present the code for your function in the R markdown document.
Include at least one example of output from your function.

# Add your solution here

Part 4: Apply functions and plot

Part 4A: Tasks

In this part, we will apply the functions we wrote in Part 1 to our rainfall data starting with our wrangled df data from Part 2.

First, filter for only years including 2014 and onwards.
For a given city and for a given year, calculate the sample mean (using your function sample_mean()), the sample standard deviation (using your function sample_sd()), and a 95% confidence interval for the average rainfall (using your function calculate_CI()). Specifically, you should add two columns in this summarized dataset: a column titled lower_bound and a column titled upper_bound containing the lower and upper bounds for you CI that you calculated (using your function calculate_CI()).
Call this summarized dataset rain_df.

# Add your solution here

Part 4B: Tasks

Using the rain_df, plots the estimates of mean rainfall and the 95% confidence intervals on the same plot. There should be a separate faceted plot for each city. Think about using ggplot() with both geom_point() (and geom_line() to connect the points) for the means and geom_errorbar() for the lower and upper bounds of the confidence interval. Check https://r-graphics.org/recipe-annotate-error-bar and or the official documentation https://ggplot2.tidyverse.org/reference/geom_linerange.html for examples of how to use geom_errorbar().

# Add your solution here

R session information

options(width = 120)
sessioninfo::session_info()

─ Session info ───────────────────────────────────────────────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.4.1 (2024-06-14)
 os       macOS Sonoma 14.5
 system   aarch64, darwin20
 ui       X11
 language (EN)
 collate  en_US.UTF-8
 ctype    en_US.UTF-8
 tz       America/New_York
 date     2024-08-20
 pandoc   3.2 @ /opt/homebrew/bin/ (via rmarkdown)

─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────────────────
 package     * version date (UTC) lib source
 cli           3.6.3   2024-06-21 [1] CRAN (R 4.4.0)
 colorout    * 1.3-0.2 2024-05-03 [1] Github (jalvesaq/colorout@c6113a2)
 colorspace    2.1-1   2024-07-26 [1] CRAN (R 4.4.0)
 digest        0.6.36  2024-06-23 [1] CRAN (R 4.4.0)
 dplyr       * 1.1.4   2023-11-17 [1] CRAN (R 4.4.0)
 evaluate      0.24.0  2024-06-10 [1] CRAN (R 4.4.0)
 fansi         1.0.6   2023-12-08 [1] CRAN (R 4.4.0)
 fastmap       1.2.0   2024-05-15 [1] CRAN (R 4.4.0)
 forcats     * 1.0.0   2023-01-29 [1] CRAN (R 4.4.0)
 generics      0.1.3   2022-07-05 [1] CRAN (R 4.4.0)
 ggplot2     * 3.5.1   2024-04-23 [1] CRAN (R 4.4.0)
 glue          1.7.0   2024-01-09 [1] CRAN (R 4.4.0)
 gtable        0.3.5   2024-04-22 [1] CRAN (R 4.4.0)
 here        * 1.0.1   2020-12-13 [1] CRAN (R 4.4.0)
 hms           1.1.3   2023-03-21 [1] CRAN (R 4.4.0)
 htmltools     0.5.8.1 2024-04-04 [1] CRAN (R 4.4.0)
 htmlwidgets   1.6.4   2023-12-06 [1] CRAN (R 4.4.0)
 jsonlite      1.8.8   2023-12-04 [1] CRAN (R 4.4.0)
 knitr         1.48    2024-07-07 [1] CRAN (R 4.4.0)
 lifecycle     1.0.4   2023-11-07 [1] CRAN (R 4.4.0)
 lubridate   * 1.9.3   2023-09-27 [1] CRAN (R 4.4.0)
 magrittr      2.0.3   2022-03-30 [1] CRAN (R 4.4.0)
 munsell       0.5.1   2024-04-01 [1] CRAN (R 4.4.0)
 pillar        1.9.0   2023-03-22 [1] CRAN (R 4.4.0)
 pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.4.0)
 purrr       * 1.0.2   2023-08-10 [1] CRAN (R 4.4.0)
 R6            2.5.1   2021-08-19 [1] CRAN (R 4.4.0)
 readr       * 2.1.5   2024-01-10 [1] CRAN (R 4.4.0)
 rlang         1.1.4   2024-06-04 [1] CRAN (R 4.4.0)
 rmarkdown     2.27    2024-05-17 [1] CRAN (R 4.4.0)
 rprojroot     2.0.4   2023-11-05 [1] CRAN (R 4.4.0)
 rstudioapi    0.16.0  2024-03-24 [1] CRAN (R 4.4.0)
 scales        1.3.0   2023-11-28 [1] CRAN (R 4.4.0)
 sessioninfo   1.2.2   2021-12-06 [1] CRAN (R 4.4.0)
 stringi       1.8.4   2024-05-06 [1] CRAN (R 4.4.0)
 stringr     * 1.5.1   2023-11-14 [1] CRAN (R 4.4.0)
 tibble      * 3.2.1   2023-03-20 [1] CRAN (R 4.4.0)
 tidyr       * 1.3.1   2024-01-24 [1] CRAN (R 4.4.0)
 tidyselect    1.2.1   2024-03-11 [1] CRAN (R 4.4.0)
 tidyverse   * 2.0.0   2023-02-22 [1] CRAN (R 4.4.0)
 timechange    0.3.0   2024-01-18 [1] CRAN (R 4.4.0)
 tzdb          0.4.0   2023-05-12 [1] CRAN (R 4.4.0)
 utf8          1.2.4   2023-10-22 [1] CRAN (R 4.4.0)
 vctrs         0.6.5   2023-12-01 [1] CRAN (R 4.4.0)
 withr         3.0.0   2024-01-16 [1] CRAN (R 4.4.0)
 xfun          0.46    2024-07-18 [1] CRAN (R 4.4.0)
 yaml          2.3.10  2024-07-26 [1] CRAN (R 4.4.0)

 [1] /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────