---
title: "Air pollution data"
author: "Julian Faraway"
date: "`r format(Sys.time(), '%d %B %Y')`"
output:
  html_document:
    keep_md: yes
---

```{r echo=FALSE}
library(knitr)
opts_chunk$set(comment=NA, warning=FALSE, fig.path="Figs/")
```


# Set up

Load in the tidyverse

```{r}
library(tidyverse)
```

# Data

We read in the data from Station 8 in Ulaanbaatar:

```{r}
ub8 = read_csv("ub8.csv")
ub8
```

We need to replace all the 99999 values with NA. The `mutate_at` command is useful here:

- we specify the range of variables as `SO2:O3`
- we apply a function to these variables in the second argument
- the `~` indicates that the following should be considered a function
- the `.` is the argument to that function
- the `na_if` command replaces a specified value with NA

```{r}
ub8 %>% mutate_at(vars(SO2:O3), ~na_if(.,99999)) -> ub8
ub8
```

The [lubridate](https://lubridate.tidyverse.org) is good for manipulating dates and times. We construct a date time variable:

```{r}
library(lubridate)
ub8 %>% mutate(datetime = make_datetime(year=year,month=month,day=day,hour=obs_time, tz="Asia/Ulaanbaatar")) -> ub8
ub8
```

Since the Medical University data starts in 2014, we filter older data out. We check the proportion of
missing values in the measures:

```{r}
ub8 %>% filter(year >= 2014) ->ub8
ub8 %>% select(SO2:O3) %>% summarise_all(~mean(is.na(.)))
```

- We see that all the PM25 and CO2 data is missing
- There is a substantial proportion of other missing data

## Exercises

1. Is the data from 2018 more complete?

2. What is the mean value of SO2 in 2014?


# Plotting

The basic line plot

```{r}
ggplot(ub8, aes(datetime, NO2)) + geom_line() 
```

Averaged over days

```{r}
ub8 %>% group_by(year,month,day) %>% summarise_at(vars(SO2:O3),~mean(.,na.rm=TRUE)) %>% ungroup() -> ub8day
ub8day %>% mutate(doy = make_date(year=year,month=month,day=day)) -> ub8day
ub8day
```

- We group by day (we need all unique combinations of year, month and day)
- We use `summarise_at` to take the means of all the air pollution variables
- Some of the values are missing so we specify `na.rm=TRUE`
- In the second line, we create a new date variable

And plotted with an appropriate smooth:

```{r}
ggplot(ub8day, aes(doy, NO2)) + geom_point(alpha=0.5) + stat_smooth(method="loess",span=0.05,se=FALSE)
```

PM10 works better on a log scale

```{r}
ggplot(ub8day, aes(doy, PM10)) + geom_point(alpha=0.5) + scale_y_log10() + stat_smooth(method="loess",span=0.1,se=FALSE)
```

## Exercises

1. Make a plot showing the relationship between NO2 and SO2.

2. Create a dataset where the variables are averaged over months, not days.

3. In the smoothed plot of NO2, try setting the `span` to a larger value of 0.25. Now try a smaller value of 0.01.
What do you think is best?

# Save for later


```{r}
save(ub8day,file="ub8day.rda")
```



