---
title: "Basic 'kendallknight' usage"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{details}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

library(kendallknight)
```

*For the full details about the implementation, please check
Sepulveda MV (2025) Kendallknight: An R package for efficient implementation of
Kendall's correlation coefficient computation. PLoS One 20(6): e0326090. 
https://doi.org/10.1371/journal.pone.0326090*


The `kendallknight` package is exclusively focused on the Kendall's correlation
coefficient and provides additional functions to test the statistical
significance of the computed correlation not available in other packages, which
is particularly useful in econometric and statistical contexts.

The `kendallknight` package is available on CRAN and can be installed using the
following command:

```r
# CRAN
install.packages("kendallknight")

# GitHub
remotes::install_github("pachadotdev/kendallknight")
```

As an illustrative exercise we can explore the question 'is there a relationship
between the number of computer science doctorates awarded in the United States
and the total revenue generated by arcades?' Certainly, this question is about
a numerical exercise and not about causal mechanisms.

The following table obtained from @vigen2015 can be used to illustrate the usage of
the `kendallknight` package:

| Year| Computer science doctorates awarded in the US| Total revenue generated by arcades|
|----:|---------------------------------------------:|----------------------------------:|
| 2000|                                           861|                              1.196|
| 2001|                                           830|                              1.176|
| 2002|                                           809|                              1.269|
| 2003|                                           867|                              1.240|
| 2004|                                           948|                              1.307|
| 2005|                                          1129|                              1.435|
| 2006|                                          1453|                              1.601|
| 2007|                                          1656|                              1.654|
| 2008|                                          1787|                              1.803|
| 2009|                                          1611|                              1.734|

The `kendall_cor()` function can be used to compute the Kendall's correlation
coefficient:

```
library(kendallknight)

kendall_cor(arcade$doctorates, arcade$revenue)

[1] 0.8222222
```

The `kendall_cor_test()` function can be used to test the null hypothesis that
the Kendall's correlation coefficient is zero:

```
kendall_cor_test(
  arcade$doctorates,
  arcade$revenue,
  conf.level = 0.8,
  alternative = "greater"
)

        Kendall's rank correlation tau

data:  arcade$doctorates and arcade$revenue
tau = 0.82222, p-value = 0.0001788
alternative hypothesis: true tau is greater than 0
80 percent confidence interval:
 0.5038182 1.0000000
```

One important difference with base R implementation is that this implementation
allows to obtain confidence intervals for different confidence levels (e.g.,
95\%, 90\%, etc).

With the obtained $p$-value and a significance level of 80\% (the default
is 95\%), the null hypothesis is rejected for the two-tailed test
($H_0: \tau = 0$ versus $H_1: \neq 0$, the default option) and the greater than
one-tailed test ($H_0: \tau = 0$ versus $H_1: \tau > 0$) but not for the lower
than one-tailed test ($H_0: \tau = 0$ versus $H_1: \tau < 0$). This suggests the
correlation is positive (e.g., more doctorates are associated with more revenue
generated by arcades). In other words, these three tests tell us that the
empirical evidence from this dataset provides three answers to the research
questions:

1. Is there any relationship? Yes, more doctorates are associated with more
   revenue generated by arcades.
2. Is there a positive relationship? Yes, more doctorates are associated with
    more revenue generated by arcades.
3. Is there a negative relationship? No, more doctorates are not associated with
    less revenue generated by arcades.

With base R or `Kendall`, an equivalent result can be obtained with the
following code:

```
cor.test(arcade$doctorates, arcade$revenue, method = "kendall")

        Kendall's rank correlation tau

data:  arcade$doctorates and arcade$revenue
T = 41, p-value = 0.0003577
alternative hypothesis: true tau is not equal to 0
sample estimates:
      tau 
0.8222222 
```

```
Kendall::Kendall(arcade$doctorates, arcade$revenue)

tau = 0.822, 2-sided pvalue =0.0012822
```

In an Econometric context, the current implementation is particularly useful to
compute the pseudo-$R^2$ statistic defined as the squared Kendall correlation in
the context of (Quasi) Poisson regression with fixed effects
[@santos;@sepulveda]. A local test reveals how the pseudo-$R^2$ computation
time drops from fifty to one percent of the time
required to compute the model coefficients by using the `fepois()` function from
the `lfe` package [@gaure] and a dataset containing fifteen thousand rows
[@yotov]:

```
library(tradepolicy)
library(lfe)

data8694 <- subset(agtpa_applications, year %in% seq(1986, 1994, 4))

fit <- fepois(
  trade ~ dist + cntg + lang + clny + rta |
    as.factor(paste0(exporter, year)) +
    as.factor(paste0(importer, year)),
  data = data8694
)

psr <- (cor(data8694$trade, fit$fitted.values, method = "kendall"))^2

psr2 <- (kendall_cor(data8694$trade, fit$fitted.values))^2

c("kendallknight" = psr, "base R" = psr2)

kendallknight        base R 
     0.263012      0.263012 
```

Comparing the model fitting time versus the correlation computation time,
we can see that the `kendallknight` package becomes relevant:

|Operation                    |  Time| Pseudo $R^2$ / Model fitting|
|:----------------------------|-----:|----------------------------:|
|Model fitting                | 3.75s|                             |
|Pseudo-$R^2$ (base R)        | 1.78s|                       47.58%|
|Pseudo-$R^2$ (kendallknight) | 0.02s|                        0.51%|

For simulated datasets with variables "x" and "y" created with `rnorm()` and
`rpois()` we observe a increasingly time differences as the number of
observations increases:

| No. of observations | kendallknight median time (s) | Kendall median time (s) | Base R median time (s) |
|:-------------------:|:-----------------------------:|:-----------------------:|:----------------------:|
|       10,000        |             0.013             |           1.0           |           4            |
|       20,000        |             0.026             |           3.9           |           16           |
|       30,000        |             0.040             |           8.7           |           36           |
|       40,000        |             0.056             |          15.6           |           64           |
|       50,000        |             0.071             |          24.2           |          100           |
|       60,000        |             0.088             |          34.8           |          144           |
|       70,000        |             0.104             |          47.5           |          196           |
|       80,000        |             0.123             |          61.9           |          256           |
|       90,000        |             0.137             |          78.2           |          324           |
|       100,000       |             0.153             |          96.4           |          399           |
