Statistique descriptive

class: center, middle, inverse, title-slide

.title[
# Statistique descriptive
]
.author[
### Traitement des données
]
.date[
### INTECHMER - CT2 GEM/PVRM
]

---

## Premier problème : décrire les données

On parle de **Statistique descriptive** ou **exploratoire**

### Objectifs

- Résumer l'information contenue dans les données
- Faire ressortir des éléments intéressants
- Poser des hypothèses sur des phénomènes potentiellement existant dans les données

### Outils
  
- Description numérique (moyenne, occurrences, corrélation...)
- Description graphique (histogramme, diagramme en barres, nuage de points...)

---
## Données exemple - `tips`

``` r
library(tidyverse)
data = read_delim("tips.csv", delim = ",")
```

<div class="datatables html-widget html-fill-item" id="htmlwidget-f6ced37180dff6ba92b0" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-f6ced37180dff6ba92b0">{"x":{"filter":"none","vertical":false,"data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100","101","102","103","104","105","106","107","108","109","110","111","112","113","114","115","116","117","118","119","120","121","122","123","124","125","126","127","128","129","130","131","132","133","134","135","136","137","138","139","140","141","142","143","144","145","146","147","148","149","150","151","152","153","154","155","156","157","158","159","160","161","162","163","164","165","166","167","168","169","170","171","172","173","174","175","176","177","178","179","180","181","182","183","184","185","186","187","188","189","190","191","192","193","194","195","196","197","198","199","200","201","202","203","204","205","206","207","208","209","210","211","212","213","214","215","216","217","218","219","220","221","222","223","224","225","226","227","228","229","230","231","232","233","234","235","236","237","238","239","240","241","242","243","244"],[16.99,10.34,21.01,23.68,24.59,25.29,8.77,26.88,15.04,14.78,10.27,35.26,15.42,18.43,14.83,21.58,10.33,16.29,16.97,20.65,17.92,20.29,15.77,39.42,19.82,17.81,13.37,12.69,21.7,19.65,9.550000000000001,18.35,15.06,20.69,17.78,24.06,16.31,16.93,18.69,31.27,16.04,17.46,13.94,9.68,30.4,18.29,22.23,32.4,28.55,18.04,12.54,10.29,34.81,9.94,25.56,19.49,38.01,26.41,11.24,48.27,20.29,13.81,11.02,18.29,17.59,20.08,16.45,3.07,20.23,15.01,12.02,17.07,26.86,25.28,14.73,10.51,17.92,27.2,22.76,17.29,19.44,16.66,10.07,32.68,15.98,34.83,13.03,18.28,24.71,21.16,28.97,22.49,5.75,16.32,22.75,40.17,27.28,12.03,21.01,12.46,11.35,15.38,44.3,22.42,20.92,15.36,20.49,25.21,18.24,14.31,14,7.25,38.07,23.95,25.71,17.31,29.93,10.65,12.43,24.08,11.69,13.42,14.26,15.95,12.48,29.8,8.52,14.52,11.38,22.82,19.08,20.27,11.17,12.26,18.26,8.51,10.33,14.15,16,13.16,17.47,34.3,41.19,27.05,16.43,8.35,18.64,11.87,9.779999999999999,7.51,14.07,13.13,17.26,24.55,19.77,29.85,48.17,25,13.39,16.49,21.5,12.66,16.21,13.81,17.51,24.52,20.76,31.71,10.59,10.63,50.81,15.81,7.25,31.85,16.82,32.9,17.89,14.48,9.6,34.63,34.65,23.33,45.35,23.17,40.55,20.69,20.9,30.46,18.15,23.1,15.69,19.81,28.44,15.48,16.58,7.56,10.34,43.11,13,13.51,18.71,12.74,13,16.4,20.53,16.47,26.59,38.73,24.27,12.76,30.06,25.89,48.33,13.27,28.17,12.9,28.15,11.59,7.74,30.14,12.16,13.42,8.58,15.98,13.42,16.27,10.09,20.45,13.28,22.12,24.01,15.69,11.61,10.77,15.53,10.07,12.6,32.83,35.83,29.03,27.18,22.67,17.82,18.78],[1.01,1.66,3.5,3.31,3.61,4.71,2,3.12,1.96,3.23,1.71,5,1.57,3,3.02,3.92,1.67,3.71,3.5,3.35,4.08,2.75,2.23,7.58,3.18,2.34,2,2,4.3,3,1.45,2.5,3,2.45,3.27,3.6,2,3.07,2.31,5,2.24,2.54,3.06,1.32,5.6,3,5,6,2.05,3,2.5,2.6,5.2,1.56,4.34,3.51,3,1.5,1.76,6.73,3.21,2,1.98,3.76,2.64,3.15,2.47,1,2.01,2.09,1.97,3,3.14,5,2.2,1.25,3.08,4,3,2.71,3,3.4,1.83,5,2.03,5.17,2,4,5.85,3,3,3.5,1,4.3,3.25,4.73,4,1.5,3,1.5,2.5,3,2.5,3.48,4.08,1.64,4.06,4.29,3.76,4,3,1,4,2.55,4,3.5,5.07,1.5,1.8,2.92,2.31,1.68,2.5,2,2.52,4.2,1.48,2,2,2.18,1.5,2.83,1.5,2,3.25,1.25,2,2,2,2.75,3.5,6.7,5,5,2.3,1.5,1.36,1.63,1.73,2,2.5,2,2.74,2,2,5.14,5,3.75,2.61,2,3.5,2.5,2,2,3,3.48,2.24,4.5,1.61,2,10,3.16,5.15,3.18,4,3.11,2,2,4,3.55,3.68,5.65,3.5,6.5,3,5,3.5,2,3.5,4,1.5,4.19,2.56,2.02,4,1.44,2,5,2,2,4,2.01,2,2.5,4,3.23,3.41,3,2.03,2.23,2,5.16,9,2.5,6.5,1.1,3,1.5,1.44,3.09,2.2,3.48,1.92,3,1.58,2.5,2,3,2.72,2.88,2,3,3.39,1.47,3,1.25,1,1.17,4.67,5.92,2,2,1.75,3],["Female","Male","Male","Male","Female","Male","Male","Male","Male","Male","Male","Female","Male","Male","Female","Male","Female","Male","Female","Male","Male","Female","Female","Male","Male","Male","Male","Male","Male","Female","Male","Male","Female","Female","Male","Male","Male","Female","Male","Male","Male","Male","Male","Male","Male","Male","Male","Male","Male","Male","Male","Female","Female","Male","Male","Male","Male","Female","Male","Male","Male","Male","Male","Male","Male","Male","Female","Female","Male","Male","Male","Female","Female","Female","Female","Male","Male","Male","Male","Male","Male","Male","Female","Male","Male","Female","Male","Male","Male","Male","Male","Male","Female","Female","Female","Male","Male","Male","Male","Male","Female","Female","Female","Female","Female","Male","Male","Male","Male","Female","Male","Female","Male","Male","Female","Female","Male","Female","Female","Female","Male","Female","Male","Male","Female","Female","Male","Female","Female","Male","Male","Female","Female","Female","Female","Female","Female","Female","Male","Female","Female","Male","Male","Female","Female","Female","Female","Female","Male","Male","Male","Male","Male","Male","Male","Female","Male","Female","Female","Male","Male","Male","Female","Male","Female","Male","Male","Male","Female","Female","Male","Male","Male","Male","Male","Male","Male","Male","Female","Male","Male","Male","Male","Male","Male","Male","Female","Male","Female","Male","Male","Female","Male","Male","Male","Male","Male","Female","Female","Male","Male","Female","Female","Female","Male","Female","Male","Male","Male","Female","Male","Male","Male","Female","Female","Female","Male","Male","Male","Female","Male","Female","Male","Female","Male","Female","Female","Male","Male","Female","Male","Male","Male","Male","Male","Male","Male","Male","Female","Male","Female","Male","Male","Female"],["No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","Yes","No","Yes","No","Yes","Yes","Yes","Yes","No","No","No","Yes","No","Yes","No","No","Yes","Yes","No","No","Yes","No","No","No","Yes","No","No","Yes","No","No","No","No","No","No","Yes","No","Yes","Yes","No","Yes","Yes","Yes","Yes","No","Yes","Yes","Yes","Yes","No","Yes","Yes","Yes","No","Yes","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","Yes","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","No","Yes","No","No","No","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","No","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","No","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","No","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","Yes","No","Yes","Yes","Yes","No","No","Yes","Yes","Yes","No","No","Yes","No","Yes","Yes","No","No","Yes","Yes","No","No"],["Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Fri","Fri","Fri","Fri","Fri","Fri","Fri","Fri","Fri","Fri","Fri","Fri","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sun","Sun","Sun","Sun","Sun","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sat","Sat","Sat","Sat","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Sun","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Thur","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Fri","Fri","Fri","Fri","Fri","Fri","Fri","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Sat","Thur"],["Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Lunch","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner","Dinner"],[2,3,3,2,4,4,2,4,2,2,2,4,2,4,2,2,3,3,3,3,2,2,2,4,2,4,2,2,2,2,2,4,2,4,2,3,3,3,3,3,3,2,2,2,4,2,2,4,3,2,2,2,4,2,4,2,4,2,2,4,2,2,2,4,3,3,2,1,2,2,2,3,2,2,2,2,2,4,2,2,2,2,1,2,2,4,2,2,2,2,2,2,2,2,2,4,2,2,2,2,2,2,3,2,2,2,2,2,2,2,2,1,3,2,3,2,4,2,2,4,2,2,2,2,2,6,2,2,2,3,2,2,2,2,2,2,2,2,2,2,2,6,5,6,2,2,3,2,2,2,2,2,3,4,4,5,6,4,2,4,4,2,3,2,2,3,2,4,2,2,3,2,2,2,2,2,2,2,2,2,4,2,3,4,2,5,3,5,3,3,2,2,2,2,2,2,2,4,2,2,3,2,2,2,4,3,3,4,2,2,3,4,4,2,3,2,5,2,2,4,2,2,1,3,2,2,2,4,2,2,4,3,2,2,2,2,2,2,3,3,2,2,2,2]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>total_bill<\/th>\n      <th>tip<\/th>\n      <th>sex<\/th>\n      <th>smoker<\/th>\n      <th>day<\/th>\n      <th>time<\/th>\n      <th>size<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"p","columnDefs":[{"className":"dt-right","targets":[1,2,7]},{"orderable":false,"targets":0},{"name":" ","targets":0},{"name":"total_bill","targets":1},{"name":"tip","targets":2},{"name":"sex","targets":3},{"name":"smoker","targets":4},{"name":"day","targets":5},{"name":"time","targets":6},{"name":"size","targets":7}],"order":[],"autoWidth":false,"orderClasses":false}},"evals":[],"jsHooks":[]}</script>

---
class: center, middle, inverse

## Variable quantitative

---

## Variable quantitative
  
- Moyenne `$\bar{x}$`
$$
  \bar{x} = \frac{1}{n} \sum_{i=1}^n x_i
$$
  
- Variance 
$$
  \sigma^2(x) = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2
$$
- Ecart-type `$\sigma(x) = \sqrt{\sigma^2(x)}$`

---

## Variable quantitative
  
- Médiane `$med(x)$` : valeur permettant de séparer les observations ordonnées prises par `$x$` en 2 groupes de même taille
$$
  med(x) = m | P(x \le m) = .5
$$
  - si `$n$` est impair : `$med(x) = x_{(n + 1) / 2}$`
  - si `$n$` est pair : `$med(x) = \frac{x_{n/2} + x_{n/2 + 1}}{2}$`
  
- Quantile `$q_p(x)$` : valeur pour laquelle une proportion `$p$` d'observations sont inférieures
$$
  q_p(x) = q | P(x \le q) = p
$$
  - Quartiles `$Q1$` et `$Q3$` : respectivement 25% et 75% (utilisés dans les boîtes à moustaches)
  - Quantiles usuels : `$.01$` (1%), `$.1$` (10%), `$.9$` (90%) et `$.99$` (99%)

---

## Variable quantitative

Exemple : montant payé par table

### Représentation numérique

<table class="table" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> Statistique </th>
   <th style="text-align:right;"> Valeur Stat </th>
   <th style="text-align:left;"> Quantile </th>
   <th style="text-align:right;"> Valeur Quantile </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Moyenne </td>
   <td style="text-align:right;"> 19.79 </td>
   <td style="text-align:left;"> 1 % </td>
   <td style="text-align:right;"> 7.25 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Ecart-Type </td>
   <td style="text-align:right;"> 8.90 </td>
   <td style="text-align:left;"> 5 % </td>
   <td style="text-align:right;"> 9.56 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Variance </td>
   <td style="text-align:right;"> 79.25 </td>
   <td style="text-align:left;"> 10 % </td>
   <td style="text-align:right;"> 10.34 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Médiane </td>
   <td style="text-align:right;"> 17.80 </td>
   <td style="text-align:left;"> 90 % </td>
   <td style="text-align:right;"> 32.24 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Minimum </td>
   <td style="text-align:right;"> 3.07 </td>
   <td style="text-align:left;"> 95 % </td>
   <td style="text-align:right;"> 38.06 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Maximum </td>
   <td style="text-align:right;"> 50.81 </td>
   <td style="text-align:left;"> 99 % </td>
   <td style="text-align:right;"> 48.23 </td>
  </tr>
</tbody>
</table>

### A regarder aussi :

- Si divergence moyenne et médiane, valeurs extrêmes présentes
  - Déséquilibre de la répartition des valeurs 
- Présence de valeurs aberrantes (nommés **outliers**)

---

## Variable quantitative
  
### Représentation graphique 
  
Histogramme

---

## Variable quantitative
  
### Représentation graphique 
  
Boîte à moustaches

---

## Variable quantitative - description numérique avec R

``` r
mean(data$total_bill) # Moyenne
```

```
## [1] 19.78594
```

``` r
var(data$total_bill) # Variance
```

```
## [1] 79.25294
```

``` r
sd(data$total_bill) # Ecart-type
```

```
## [1] 8.902412
```

``` r
min(data$total_bill) # Minimum
```

```
## [1] 3.07
```

``` r
max(data$total_bill) # Maximum
```

```
## [1] 50.81
```

---

## Variable quantitative - description numérique avec R

``` r
range(data$total_bill) # Min et Max en même temps
```

```
## [1]  3.07 50.81
```

``` r
median(data$total_bill) # Médiane
```

```
## [1] 17.795
```

``` r
summary(data$total_bill) # Plusieurs choses en même temps
```

```
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##    3.07   13.35   17.80   19.79   24.13   50.81
```

``` r
quantile(data$total_bill, c(.25, .75)) # Q1 et Q3
```

```
##     25%     75% 
## 13.3475 24.1275
```

``` r
quantile(data$total_bill, c(.01, .05, .10, .90, .95, .99)) # Quantiles usuels
```

```
##      1%      5%     10%     90%     95%     99% 
##  7.2500  9.5575 10.3400 32.2350 38.0610 48.2270
```

---

## Variable quantitative - description numérique avec R

`summary()` applicable à plusieurs variables en même temps (intéressant dans une phase d'analyse rapide)

``` r
data %>% select(total_bill, tip, size) %>% summary()
```

```
##    total_bill         tip              size     
##  Min.   : 3.07   Min.   : 1.000   Min.   :1.00  
##  1st Qu.:13.35   1st Qu.: 2.000   1st Qu.:2.00  
##  Median :17.80   Median : 2.900   Median :2.00  
##  Mean   :19.79   Mean   : 2.998   Mean   :2.57  
##  3rd Qu.:24.13   3rd Qu.: 3.562   3rd Qu.:3.00  
##  Max.   :50.81   Max.   :10.000   Max.   :6.00
```

Moyenne calculable sur plusieurs variables en même temps

``` r
data %>% select(total_bill, tip, size) %>% 
  summarise_all(mean)
```

```
## # A tibble: 1 × 3
##   total_bill   tip  size
##        <dbl> <dbl> <dbl>
## 1       19.8  3.00  2.57
```

---

## Variable quantitative -- histogramme avec R

``` r
ggplot(data, aes(x = total_bill)) +
  geom_histogram()
```

---

## Variable quantitative -- histogramme avec R

- Trop (ou pas assez) d'intervalles peut biaiser l'analyse
- Modification avec le paramètre `bins`

``` r
ggplot(data, aes(x = total_bill)) +
  geom_histogram(bins = 10)
```

---

## Variable quantitative -- histogramme avec R

- Densité estimée par noyau
- Intéressant car permet de plus facilement voir la tendance globale

``` r
ggplot(data, aes(x = total_bill)) +
  geom_density()
```

---

## Variable quantitative -- boîte à moustaches avec R

``` r
ggplot(data, aes(x = total_bill)) +
  geom_boxplot()
```

---
class: middle, center, inverse

## Variable qualitative

---

## Variable qualitative
  
### Nominale
  
- Modalités de la variable `$x$` : `$m_j$` (avec `$j=1,...,p$`)
- Effectif (ou occurrences) d'une modalité `$n_j$` : nombre d'individus ayant la modalité `$m_j$`
  - Fréquence d'une modalité `$f_j$`
$$
f_j = \frac{n_j}{n}
$$

### Ordinale

- Effectif cumulé `$n_j^{cum}$` : nombre d'individus ayant une modalité entre `$n_1$` et `$n_j$`
  - Fréquence cumulée

`$$n_j^{cum} = \sum_{k=1}^j n_k \mbox{ and } f_j^{cum} = \sum_{k=1}^j f_k$$`
  
---

## Variable qualitative
  
Exemple : Jour de la semaine (*ordinale* de plus)

### Représentation numérique

<table class="table" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> Modalités </th>
   <th style="text-align:right;"> Effectifs </th>
   <th style="text-align:right;"> Eff. cum. </th>
   <th style="text-align:right;"> Fréquences </th>
   <th style="text-align:right;"> Fréq. cum. </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Fri </td>
   <td style="text-align:right;"> 19 </td>
   <td style="text-align:right;"> 19 </td>
   <td style="text-align:right;"> 0.08 </td>
   <td style="text-align:right;"> 0.08 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Sat </td>
   <td style="text-align:right;"> 87 </td>
   <td style="text-align:right;"> 106 </td>
   <td style="text-align:right;"> 0.36 </td>
   <td style="text-align:right;"> 0.43 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Sun </td>
   <td style="text-align:right;"> 76 </td>
   <td style="text-align:right;"> 182 </td>
   <td style="text-align:right;"> 0.31 </td>
   <td style="text-align:right;"> 0.75 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Thur </td>
   <td style="text-align:right;"> 62 </td>
   <td style="text-align:right;"> 244 </td>
   <td style="text-align:right;"> 0.25 </td>
   <td style="text-align:right;"> 1.00 </td>
  </tr>
</tbody>
</table>

### A regarder aussi :
  
- Différence entre les proportions 
- Si modalités peu fréquentes, regroupement de modalités à envisager

---

## Variable qualitative
  
### Représentation graphique
  
Diagramme en barres

---

## Variable qualitative - description numérique avec R

``` r
data %>% group_by(day) %>% count()
```

```
## # A tibble: 4 × 2
## # Groups:   day [4]
##   day       n
##   <chr> <int>
## 1 Fri      19
## 2 Sat      87
## 3 Sun      76
## 4 Thur     62
```

``` r
data %>% group_by(day) %>% count() %>% ungroup() %>% 
  mutate(n_cum = cumsum(n), freq = n / sum(n), freq_cum = n_cum / sum(n))
```

```
## # A tibble: 4 × 5
##   day       n n_cum   freq freq_cum
##   <chr> <int> <int>  <dbl>    <dbl>
## 1 Fri      19    19 0.0779   0.0779
## 2 Sat      87   106 0.357    0.434 
## 3 Sun      76   182 0.311    0.746 
## 4 Thur     62   244 0.254    1
```

---

## Variable qualitative - diagramme en barres avec R

``` r
ggplot(data, aes(x = day)) +
  geom_bar()
```

---

## Variable qualitative - diagramme en barres avec R

- Si beaucoup de modalités ou noms des modalités longs, intéressant en mode horizontale

``` r
ggplot(data, aes(y = day)) +
  geom_bar()
```

---

## Variable qualitative - diagramme en barres avec R

- Modification de l'ordre des modalités possible

``` r
ggplot(data, aes(x = day)) +
  geom_bar() +
  scale_x_discrete(limits = c("Thur", "Fri", "Sat", "Sun"))
```

---

## Variable qualitative - diagramme en barres empilées avec R

- Empilement intéressant (surtout quand on va comparer)
- Noter la fonction `fct_relevel()` pour changer l'ordre d'affichage des modalités

``` r
ggplot(data, aes(x = "", fill = fct_relevel(day, c("Thur", "Fri", "Sat", "Sun")))) +
  geom_bar()
```

---

## Variable qualitative - diagramme circulaire avec R

- **ATTENTION** : à ne pas utiliser avec variable ordinale et avec trop de modalités

``` r
ggplot(data, aes(x = "", fill = sex)) +
  geom_bar(width = 1) +
  coord_polar(theta = "y")
```

---
class: middle, center, inverse

## Lien entre deux variables quantitatives

---

## Quantitative vs quantitative
  
### Covariance
$$
  cov(x,y) = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x}) (y_i - \bar{y})
$$
- Problème : non bornée et donc non exploitable

### Coefficient de corrélation linéaire (de *Pearson*)
$$
  \rho(x,y) = \frac{cov(x,y)}{\sigma^2(x) \sigma^2(y)}
$$
- Covariance des variables normalisées
- Valeurs comprises entre -1 et 1
  - `$0$` : pas de lien linéaire (autre type de lien possible)
  - `$1$` : lien positif fort (si `$x$` augmente, `$y$` augmente)
  - `$-1$` : lien négatif fort (si `$x$` augmente, `$y$` diminue)

---

## Quantitative vs quantitative
  
Exemple : Montant de la table et Pourboire

### Représentation numérique

### A regarder aussi :
  
- Présence d'**outliers** avec un comportement atypique

---

## Quantitative vs quantitative

### Représentation graphique

Nuage de points

---

## Anscombe

La visualisation est aussi importante (voire plus) que la représentation numérique !

Entre ces quatre séries :

- même moyenne et même variance pour `$x$` et `$y$`
- même coefficient de corrélation entre les deux

<table class="table" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;">  </th>
   <th style="text-align:right;"> 1 </th>
   <th style="text-align:right;"> 2 </th>
   <th style="text-align:right;"> 3 </th>
   <th style="text-align:right;"> 4 </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Moyenne(x) </td>
   <td style="text-align:right;"> 9.00 </td>
   <td style="text-align:right;"> 9.00 </td>
   <td style="text-align:right;"> 9.00 </td>
   <td style="text-align:right;"> 9.00 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Moyenne(y) </td>
   <td style="text-align:right;"> 7.50 </td>
   <td style="text-align:right;"> 7.50 </td>
   <td style="text-align:right;"> 7.50 </td>
   <td style="text-align:right;"> 7.50 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Ecart-type(x) </td>
   <td style="text-align:right;"> 3.32 </td>
   <td style="text-align:right;"> 3.32 </td>
   <td style="text-align:right;"> 3.32 </td>
   <td style="text-align:right;"> 3.32 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Ecart-type(y) </td>
   <td style="text-align:right;"> 2.03 </td>
   <td style="text-align:right;"> 2.03 </td>
   <td style="text-align:right;"> 2.03 </td>
   <td style="text-align:right;"> 2.03 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Covariance </td>
   <td style="text-align:right;"> 5.50 </td>
   <td style="text-align:right;"> 5.50 </td>
   <td style="text-align:right;"> 5.50 </td>
   <td style="text-align:right;"> 5.50 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Corrélation </td>
   <td style="text-align:right;"> 0.82 </td>
   <td style="text-align:right;"> 0.82 </td>
   <td style="text-align:right;"> 0.82 </td>
   <td style="text-align:right;"> 0.82 </td>
  </tr>
</tbody>
</table>

---

## Anscombe

---

## Quantitative vs quantitative - description numérique avec R

``` r
cov(data$total_bill, data$tip)
```

```
## [1] 8.323502
```

``` r
cor(data$total_bill, data$tip)
```

```
## [1] 0.6757341
```

``` r
cor(data %>% select(total_bill, tip, size))
```

```
##            total_bill       tip      size
## total_bill  1.0000000 0.6757341 0.5983151
## tip         0.6757341 1.0000000 0.4892988
## size        0.5983151 0.4892988 1.0000000
```
---

## Quantitative vs quantitative - nuage de points avec R

``` r
ggplot(data, aes(x = total_bill, y = tip)) +
  geom_point()
```

---
class: middle, center, inverse

## Lien entre deux variables qualitatives

---

## Qualitative vs qualitative

### Table de contingence

- Croisement des 2 ensembles de modalités, avec le nombre d'individus ayant chaque couple de modalités
- `$n_{ij}$` : Nombre d'observations ayant la modalité `$i$` pour `$x$` et `$j$` pour `$y$`  
- `$n_{i.}$` : Effectif marginal (nombre d'observations ayant la modalité `$i$` pour `$x$`)
- `$n_{.j}$` : Effectif marginal (nombre d'observations ayant la modalité `$j$` pour `$y$`)
- `$n_{..}$` : Effectif total

<div class="datatables html-widget html-fill-item" id="htmlwidget-527c602ae75c8c3d8d74" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-527c602ae75c8c3d8d74">{"x":{"filter":"none","vertical":false,"data":[["1","...","i","...","k","Total"],["","","","","",""],["","","","","",""],["","","n_ij","","","n_.j"],["","","","","",""],["","","","","",""],["","","n_i.","","","n_.."]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>1<\/th>\n      <th>...<\/th>\n      <th>j<\/th>\n      <th>...<\/th>\n      <th>l<\/th>\n      <th>Total<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"","ordering":false,"columnDefs":[{"orderable":false,"targets":0},{"name":" ","targets":0},{"name":"1","targets":1},{"name":"...","targets":2},{"name":"j","targets":3},{"name":"...","targets":4},{"name":"l","targets":5},{"name":"Total","targets":6}],"order":[],"autoWidth":false,"orderClasses":false,"rowCallback":"function(row, data, displayNum, displayIndex, dataIndex) {\nvar value=data[6]; $(this.api().cell(row, 6).node()).css({'background-color':'lightblue'});\nvar value=data[0]; $(row).css({'background-color':$.inArray(dataIndex + 1, [6]) >= 0 ? \"lightblue\" : null});\nvar value=data[0]; $(this.api().cell(row, 0).node()).css({'background-color':$.inArray(dataIndex + 1, [1]) >= 0 ? \"lightgray\" : $.inArray(dataIndex + 1, [2]) >= 0 ? \"lightgray\" : $.inArray(dataIndex + 1, [3]) >= 0 ? \"lightgray\" : $.inArray(dataIndex + 1, [4]) >= 0 ? \"lightgray\" : $.inArray(dataIndex + 1, [5]) >= 0 ? \"lightgray\" : null});\n}"}},"evals":["options.rowCallback"],"jsHooks":[]}</script>

---

## Qualitative vs qualitative

### Profils lignes et colonnes
- Distribution d'une variable conditionnellement aux modalités de l'autre
  
### Profil ligne
- Pour une ligne `$i$` : `$\frac{n_{ij}}{n_{i.}}$`
- Somme des valeurs en lignes = 100%

### Profil colonne
- Pour une colonne `$j$` : `$\frac{n_{ij}}{n_{.j}}$`
- Somme des valeurs en colonnes = 100%

---

## Qualitative vs qualitative

Exemple : Jour de la semaine et Présence de fumeur

### Représentation numérique

### A regarder aussi :

- Couple de modalités très peu pris
- Ici aussi, regroupement de modalités à envisager éventuellement

---

## Qualitative vs qualitative

### Représentation graphique

---

## Qualitative vs qualitative

### Représentation numérique

Profils colonnes ici (sommes en colonnes = 100%)

---

## Qualitative vs qualitative

### Représentation graphique

Profils colonnes

---

## Qualitative vs qualitative

### Représentation numérique

Profils lignes ici (sommes en lignes = 100%)

---

## Qualitative vs qualitative

### Représentation graphique

Profils lignes

---

## Qualitative vs qualitative - description numérique avec R

Table de contingence

``` r
table(data$day, data$smoker)
```

```
##       
##        No Yes
##   Fri   4  15
##   Sat  45  42
##   Sun  57  19
##   Thur 45  17
```

Fréquence globale

``` r
prop.table(table(data$day, data$smoker))
```

```
##       
##                No        Yes
##   Fri  0.01639344 0.06147541
##   Sat  0.18442623 0.17213115
##   Sun  0.23360656 0.07786885
##   Thur 0.18442623 0.06967213
```

---

## Qualitative vs qualitative - description numérique avec R

Profils lignes

``` r
prop.table(table(data$day, data$smoker), margin = 1)
```

```
##       
##               No       Yes
##   Fri  0.2105263 0.7894737
##   Sat  0.5172414 0.4827586
##   Sun  0.7500000 0.2500000
##   Thur 0.7258065 0.2741935
```

Profils colonnes

``` r
prop.table(table(data$day, data$smoker), margin = 2)
```

```
##       
##                No        Yes
##   Fri  0.02649007 0.16129032
##   Sat  0.29801325 0.45161290
##   Sun  0.37748344 0.20430108
##   Thur 0.29801325 0.18279570
```

---

## Qualitative vs qualitative - diagramme en barres avec R

Empilées par défaut, mais pas avec une somme à 100%

``` r
ggplot(data, aes(x = day, fill = smoker)) +
  geom_bar()
```

---

## Qualitative vs qualitative - diagramme en barres avec R

On peut les séparer, mais attention si une des modalités est beaucoup trop présente (risque d'avoir un graphique illisible)

``` r
ggplot(data, aes(x = day, fill = smoker)) +
  geom_bar(position = "dodge")
```

---

## Qualitative vs qualitative - diagramme en barres avec R

- Empilées avec une somme à 100% (donc avec des vrais profils)
- `x` et `fill` à inverser si on veut voir les autres profils

``` r
ggplot(data, aes(x = day, fill = smoker)) +
  geom_bar(position = "fill")
```

---
class: middle, center, inverse

## Lien entre une variable quantitative et une variable qualitative

---

## Qualitative vs quantitative

- Soit `$Y$` la variable qualitative à `$m$` modalités, et `$X$` la variable quantitative
- Sous-populations déterminées par les modalités de `$Y$`
- Indicateurs calculés pour chaque modalité k

`$$\bar{x_j} = \frac{1}{n_j} \sum_{i | y_i = j} x_i$$`

`$$\sigma^2(x_j) = \frac{1}{n_j} \sum {}_{i | y_i = j} (x_i - \bar{x_j})^2$$`

---

## Qualitative vs quantitative

Exemple : Montant payé et Jour de la semaine

### Représentation numérique

<table class="table" style="margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> day </th>
   <th style="text-align:right;"> Moyenne </th>
   <th style="text-align:right;"> Ecart-type </th>
   <th style="text-align:right;"> Médiane </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Fri </td>
   <td style="text-align:right;"> 17.15 </td>
   <td style="text-align:right;"> 8.30 </td>
   <td style="text-align:right;"> 15.38 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Sat </td>
   <td style="text-align:right;"> 20.44 </td>
   <td style="text-align:right;"> 9.48 </td>
   <td style="text-align:right;"> 18.24 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Sun </td>
   <td style="text-align:right;"> 21.41 </td>
   <td style="text-align:right;"> 8.83 </td>
   <td style="text-align:right;"> 19.63 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Thur </td>
   <td style="text-align:right;"> 17.68 </td>
   <td style="text-align:right;"> 7.89 </td>
   <td style="text-align:right;"> 16.20 </td>
  </tr>
</tbody>
</table>

### A regarder aussi :

- Outliers

---

## Qualitative vs quantitative

### Représentation graphique 
Boîte à moustaches

---

## Qualitative vs quantitative - description numérique avec R

``` r
data %>%
  group_by(day) %>%
  summarise(Moyenne = mean(total_bill), EcartType = sd(total_bill))
```

```
## # A tibble: 4 × 3
##   day   Moyenne EcartType
##   <chr>   <dbl>     <dbl>
## 1 Fri      17.2      8.30
## 2 Sat      20.4      9.48
## 3 Sun      21.4      8.83
## 4 Thur     17.7      7.89
```

---

## Qualitative vs quantitative - densité avec R

``` r
ggplot(data, aes(x = total_bill, color = day)) +
  geom_density()
```

---

## Qualitative vs quantitative - boîtes à moustaches avec R

Inversion de `x` et `y` possible

``` r
ggplot(data, aes(x = total_bill, y = day)) +
  geom_boxplot()
```

---
class: middle, center, inverse

## Quelques compléments

---

## Personnalisation des graphiques possible dans `ggplot()`

``` r
ggplot(data, aes(x = total_bill, 
                 y = fct_relevel(day, c("Thur", "Fri", "Sat", "Sun")), 
                 fill = fct_relevel(day, c("Thur", "Fri", "Sat", "Sun")))) +
  geom_boxplot() +
  scale_fill_manual(values = c("orchid", "orange", "steelblue", "limegreen")) +
  scale_y_discrete(limits = rev) +
  theme_minimal() + # d'autres thèmes existent
  labs(x = "Montant facture", y = "", fill = "Jour de la\nsemaine")
```

---

## Transformation de variable
  
### Quantitative en qualitative
  
- Courant de transformer une variable **quantitative** en variable **qualitative ordinale**
- Ex : Catégorie d'âge, Nombre d'enfants du foyer, ...

- Combien de modalités (*intervalles* ici) ? 
  - Taille identique des intervalles ou variable (*amplitude*) ?
  - Seuils des intervalles ?

- *Plus simple* Transformer en variable binaire : présence / absence
  - peut-être *trop* simple

---

## Transformation de variable
  
### Standardisation ou normalisation d'une variable quantitative
  
- Obligatoire pour l'utilisation de certaines méthodes statistiques

- 2 opérations sont réalisées :
  - Centrage : on retire la moyenne à chaque valeur
  - Réduction : on divise par la variance
$$
  x_{norm} = \frac{x - \bar{x}}{\sigma^2}
$$

---

## Transformation de variable

### Beaucoup d'autres envisageables
  
- Division par le total ou par le maximum
- Normalisation pour que la somme en ligne soit égale à 1
- Utiliser les rangs
- Transformation de Hellinger 
$$
y\prime_i^j = \sqrt{\frac{y_i^j}{\sum_k y_i^k}}
$$
- Calcul de `$\log(x)$` voire `$\log(1+x)$` (si valeur 0 présente par exemple)
- Plusieurs autres possibles...