import polars as pl
import scipy as sp
import numpy as npurl = "https://roualdes.sfo3.digitaloceanspaces.com/data/penguins.csv"
df = pl.read_csv(url, null_values = ["NA"]) # this is the trickdf
shape: (344, 8)
| species | island | bill_length_mm | bill_depth_mm | flipper_length_mm | body_mass_g | sex | year |
|---|---|---|---|---|---|---|---|
| str | str | f64 | f64 | i64 | i64 | str | i64 |
| "Adelie" | "Torgersen" | 39.1 | 18.7 | 181 | 3750 | "male" | 2007 |
| "Adelie" | "Torgersen" | 39.5 | 17.4 | 186 | 3800 | "female" | 2007 |
| "Adelie" | "Torgersen" | 40.3 | 18.0 | 195 | 3250 | "female" | 2007 |
| "Adelie" | "Torgersen" | null | null | null | null | null | 2007 |
| "Adelie" | "Torgersen" | 36.7 | 19.3 | 193 | 3450 | "female" | 2007 |
| … | … | … | … | … | … | … | … |
| "Chinstrap" | "Dream" | 55.8 | 19.8 | 207 | 4000 | "male" | 2009 |
| "Chinstrap" | "Dream" | 43.5 | 18.1 | 202 | 3400 | "female" | 2009 |
| "Chinstrap" | "Dream" | 49.6 | 18.2 | 193 | 3775 | "male" | 2009 |
| "Chinstrap" | "Dream" | 50.8 | 19.0 | 210 | 4100 | "male" | 2009 |
| "Chinstrap" | "Dream" | 50.2 | 18.7 | 198 | 3775 | "female" | 2009 |
bill_length = df["bill_length_mm"].to_numpy() # .cast(pl.Float64, strict=False).to_numpy()np.mean(bill_length)np.float64(nan)data = bill_length[~np.isnan(bill_length)]
mhat = np.mean(data)
shat = np.std(data)
N = np.size(data)
p = .99
t = sp.stats.t(df = N - 1).ppf(p + (1 - p)/2)mhat + t * shat / np.sqrt(N)np.float64(44.685527571236165)mhat - t * shat / np.sqrt(N)np.float64(43.15833207788664)We are 99% confident that the true mean bill length of penguins is between 43.2 and 44.7 milimeters.