---
title: Fitting GPR and generation of features for new test set with 100k light curves
author: Julian Faraway
output:
 html_document:
  toc: true
  theme: cosmo
---

```{r global_options, include=FALSE}
library(knitr)
opts_chunk$set(comment=NA, fig.path='/tmp/Figs/', warning=FALSE, message=FALSE)
```

```{r echo=FALSE}
paste("Created:",date())
```

Load in the new data. Load code from previous work.

```{r}
load("lc100k.rda")
row.names(lcdbi) <- lcdbi$id
lcdbi$id <- NULL
lcdbi$type <- factor(lcdbi$type)
source("funcs.R")
GPR <- TRUE
```

Set the kernel type to squared exponential and set the inverse width:

```{r}
kerntype <- "exponential"
wvec <- 2e-4
```

```{r}
summary(lcdbi)
```
We see no short light curves.

Investigate the range of observation. Compute the date of the first observation:

```{r}
fd <- sapply(lcdb, function(x) min(x$mjd))
min(fd)
```

We see that 53464 is indeed the start date.

```{r}
robv <- sapply(lcdb, function(x) max(x$mjd) - min(x$mjd))
max(robv)
```

Seems we have substantially shorter range of observation in this dataset
than the 2764 used previously.

Sort the lightcurves to be monotonely increasing in date and create the measurement grouping variable:

```{r}
for(i in 1:length(lcdb)){
    lc <- lcdb[[i]]
    ii <- order(lc$mjd)
    lc <- lc[ii,]
    lc$measgrp <- factor(c(0,cumsum(ifelse(diff(lc$mjd) < 1, 0, 1))))
    lcdb[[i]] <- lc
}
```

## Detection Limit

For each light curve, we compute
- proportion of observations with magnitudes greater than 20.5, 21 and 21.5
- the maximum magnitude

```{r}
dlmat <- matrix(NA,length(lcdb),4)
for(i in 1:length(lcdb)){
    lc <- lcdb[[i]]
    nobs <- nrow(lc)
    dl1 <- sum(lc$mag > 20.5)/nobs
    dl2 <- sum(lc$mag > 21)/nobs
    dl3 <- sum(lc$mag > 21.5)/nobs
    dl4 <- max(lc$mag)
    dlmat[i,] <- c(dl1, dl2, dl3, dl4)
}
colnames(dlmat) <- c("d20.5","d21","d21.5","min")
dlmat <- data.frame(dlmat)
summary(dlmat)
```

Looks like 20.5 is a suitable detection limit here.

Now ready to compute the measure.


```{r}
firstdate <- 53464
daterange <- 2222
detection.limit <- 20.5
lcdbi$id <- row.names(lcdbi)
```


```{r child="meascomp.Rmd",eval=TRUE}
```

```{r}
cmdb$wander <- NULL
cmdb$moveloc <- NULL
summary(cmdb)
```

```{r}
for(i in 1:(ncol(cmdb)-2)){
   plot(cmdb[,i] ~ cmdb$type, main=names(cmdb)[i])
   }
```

```{r}
cmdbig <- cmdb
save(cmdbig,file="featbig.rda")
```






