THE DATASET CARD

$ dataset.py profile tickets.csv --label label
PROFILE · tickets.csv
rows 525 · columns 9 · delimiter ',' · encoding utf-8 · ragged rows 0
row numbers count data rows: row 1 is the first row under the header.
read as missing: "", ?, missing, n/a, na, nan, none, null (an assumption; change it with --na)

| column | type | missing | distinct | notes |
|---|---|---|---|---|
| ticket_id | text | 0 (0.0%) | 525 | every value unique: an ID? keep it out of the features |
| customer_id | text | 0 (0.0%) | 140 |  |
| customer_name | text | 0 (0.0%) | 78 |  |
| created | date | 0 (0.0%) | 112 |  |
| channel | category | 0 (0.0%) | 4 |  |
| ip | text | 0 (0.0%) | 329 |  |
| text | text | 0 (0.0%) | 514 |  |
| response_minutes | integer | 17 (3.2%) | 145 |  |
| label | category | 0 (0.0%) | 4 |  |

duplicate rows: 0 of 525 (0.0%) — equal to an earlier row in every column (values trimmed at the ends)
without the ID-like column ticket_id: 11 duplicate rows (2.1%) — row 6 = row 5 · row 32 = row 31 · row 55 = row 54 · row 81 = row 80 · row 143 = row 142 · row 220 = row 219 · row 252 = row 251 · row 277 = row 276 · and 3 more

label: label (named with --label)
| class | rows | share |
|---|---|---|
| billing | 192 | 36.6% |
| shipping | 160 | 30.5% |
| account | 112 | 21.3% |
| other | 61 | 11.6% |
largest class ÷ smallest class: 3.15  ← imbalanced: report per-class scores, not accuracy alone

numbers:
| column | min | Q1 | median | Q3 | max | mean | sd | outside the fences |
|---|---|---|---|---|---|---|---|---|
| response_minutes | 8 | 36 | 53 | 79.25 | 4210 | 73.35 | 190.3 | 27 |
  response_minutes: fences -28.875 and 144.125 · rows 1 (162), 18 (4,210), 30 (189), 43 (168), 58 (180) and 22 more

formula: missing share = missing cells ÷ rows; a duplicate row equals an earlier row in every column; quartiles by linear interpolation between order statistics (statistics.quantiles, inclusive: the same as Excel QUARTILE.INC and R type 7); outside the fences = below Q1 − 1.5·IQR or above Q3 + 1.5·IQR, IQR = Q3 − Q1
source: Tukey (1977), Exploratory Data Analysis, Addison-Wesley (the 1.5 x IQR fences); Hyndman & Fan (1996), Sample quantiles in statistical packages, The American Statistician 50(4):361-365
limit: a value outside the fences is one to look at, not an error to delete; types are inferred from the text (95% of values must parse); exact duplicates only (near-duplicates are not found); the whole file is read into memory.

$ dataset.py agree labels-2.csv --top 6
AGREE · labels-2.csv · two labellers (ann_a, ann_b) · 525 items labelled by both
percent agreement: 454 of 525 = 86.5%
chance agreement p_e (from each labeller's own label shares): 27.8%
Cohen's kappa: 0.813  (95% bootstrap interval 0.771 to 0.855; 1,000 resamples of the items, seed 7)
Landis & Koch (1977) band: almost perfect — a convention, not a test: Landis & Koch offered the bands without evidence; what is good enough depends on the task

confusion table (rows ann_a, columns ann_b):
| ann_a \ ann_b | account | billing | other | shipping | total |
|---|---|---|---|---|---|
| account | 90 | 16 | 3 | 3 | 112 |
| billing | 13 | 174 | 3 | 2 | 192 |
| other | 2 | 0 | 57 | 2 | 61 |
| shipping | 3 | 3 | 21 | 133 | 160 |
| total | 108 | 193 | 84 | 140 | 525 |

label shares: ann_a account 21.3%, billing 36.6%, other 11.6%, shipping 30.5% · ann_b account 20.6%, billing 36.8%, other 16.0%, shipping 26.7%
where they part most: ann_a says shipping, ann_b says other: 21 · ann_a says account, ann_b says billing: 16 · ann_a says billing, ann_b says account: 13 · ann_a says account, ann_b says other: 3 · ann_a says billing, ann_b says other: 3
the disputed items (71; the first 6 shown):
  T5002 · ann_a shipping · ann_b other
  T5003 · ann_a account · ann_b other
  T5013 · ann_a billing · ann_b other
  T5015 · ann_a billing · ann_b account
  T5018 · ann_a billing · ann_b account
  T5028 · ann_a account · ann_b billing

formula: κ = (p_o − p_e) ÷ (1 − p_e); p_o = share of items given the same label; p_e = Σ_k (share of ann_a's labels in k × share of ann_b's labels in k); interval: percentile bootstrap over items
source: Cohen (1960), A coefficient of agreement for nominal scales, Educational and Psychological Measurement 20(1):37-46; bands: Landis & Koch (1977), The measurement of observer agreement for categorical data, Biometrics 33(1):159-174; interval: Efron (1979), Bootstrap methods: another look at the jackknife, Annals of Statistics 7(1):1-26
limit: kappa moves with how often each label occurs: when one label dominates, agreement can be high and kappa low (Feinstein & Cicchetti 1990); unweighted, so a near miss on an ordered scale counts as a full miss; agreement is not correctness: two labellers can agree and both be wrong.

$ dataset.py agree labels-3.csv --top 5
AGREE · labels-3.csv · 3 labellers (ann_a, ann_b, ann_c) · 150 items with 3 labels each
all 3 agree on: 89 of 150 items = 59.3%
mean pairwise agreement P̄ (the share of labeller pairs on an item that agree, averaged over items): 72.2%
chance agreement P_e (from the pooled label shares): 27.7%
Fleiss' kappa: 0.616  (95% bootstrap interval 0.541 to 0.685; 1,000 resamples of the items, seed 7)
Landis & Koch (1977) band: substantial — a convention, not a test: Landis & Koch offered the bands without evidence; what is good enough depends on the task

by label (share of all labels · its own kappa, Fleiss 1971):
  other: 14.4% · κ_j 0.407
  account: 20.2% · κ_j 0.559
  shipping: 35.6% · κ_j 0.661
  billing: 29.8% · κ_j 0.734
  the label to look at first: other (lowest κ_j)

confusion between labels (pairs of labellers on the same item who split this way):
| label | label | pairs |
|---|---|---|
| other | shipping | 50 |
| account | billing | 39 |
| account | shipping | 14 |
| account | other | 11 |
| billing | shipping | 6 |
| billing | other | 5 |

each pair of labellers (Cohen's kappa on the items both labelled):
  ann_b–ann_c: 0.483 (150 items)
  ann_a–ann_c: 0.684 (150 items)
  ann_a–ann_b: 0.687 (150 items)

the most disputed items (lowest P_i; the first 5 shown):
  T5173 · shipping 1 · billing 1 · other 1 · P_i 0.00
  T5210 · billing 1 · shipping 1 · account 1 · P_i 0.00
  T5363 · other 1 · account 1 · shipping 1 · P_i 0.00
  T5018 · billing 2 · account 1 · P_i 0.33
  T5031 · billing 2 · account 1 · P_i 0.33

formula: P_i = (Σ_j n_ij² − n) ÷ (n(n − 1)); P̄ = mean of P_i; p_j = share of all labels in j; P_e = Σ_j p_j²; κ = (P̄ − P_e) ÷ (1 − P_e); κ_j = 1 − Σ_i n_ij(n − n_ij) ÷ (N·n(n − 1)·p_j(1 − p_j)); interval: percentile bootstrap over items
source: Fleiss (1971), Measuring nominal scale agreement among many raters, Psychological Bulletin 76(5):378-382; bands: Landis & Koch (1977), The measurement of observer agreement for categorical data, Biometrics 33(1):159-174; interval: Efron (1979), Bootstrap methods: another look at the jackknife, Annals of Statistics 7(1):1-26
limit: Fleiss' kappa needs the same number of labels on every item (others are left out and counted above); with two labels an item it equals Scott's pi, not Cohen's kappa; it moves with how often each label occurs (Feinstein & Cicchetti 1990); agreement is not correctness.

$ dataset.py leak naive-train.csv naive-test.csv --label label --group customer_id --ignore ticket_id,created
LEAK · naive-train.csv (420 rows) · naive-test.csv (105 rows)
inputs compared on: customer_name, channel, ip, text, response_minutes
LEAK CHECK
  exact rows in more than one split: 0
  the same inputs (every column but label, customer_id, ticket_id, created) in more than one split: 1 — e.g. naive-train.csv rows 4 · naive-test.csv rows 2
    of those, with different labels on the two sides: 0
  groups (customer_id) in more than one split: 63 — e.g. C0101: naive-train.csv rows 1, 2 · naive-test.csv rows 1
  verdict: LEAKS — the test score will read higher than it should; fix the split before trusting any number

formula: a leak = the same row, the same inputs or the same group in more than one split; rows compared after trimming spaces at the ends
source: Kaufman, Rosset, Perlich & Stitelman (2012), Leakage in data mining: formulation, detection, and avoidance, ACM TKDD 6(4), article 15; Kapoor & Narayanan (2023), Leakage and the reproducibility crisis in machine-learning-based science, Patterns 4(9):100804
limit: exact matches only: near-duplicates, the same person under two IDs, and features computed from the whole dataset before the split are not found.

$ dataset.py split tickets.csv --label label --group customer_id --test 0.2 --seed 7 --ignore ticket_id,created --out-dir splits
SPLIT · tickets.csv → splits/tickets.train.csv (421 rows) · splits/tickets.test.csv (104 rows)
seed 7 · shares train 0.80 · test 0.20 · groups by customer_id, each group kept whole
units placed: 140 (rows with identical inputs joined to one unit: 0)

| label | all | train | test |
|---|---|---|---|
| account | 112 | 90 (21.4%) | 22 (21.2%) |
| billing | 192 | 154 (36.6%) | 38 (36.5%) |
| other | 61 | 49 (11.6%) | 12 (11.5%) |
| shipping | 160 | 128 (30.4%) | 32 (30.8%) |
| rows | 525 | 421 (80.2% of all) | 104 (19.8% of all) |

LEAK CHECK
  exact rows in more than one split: 0
  the same inputs (every column but label, customer_id, ticket_id, created) in more than one split: 0
    of those, with different labels on the two sides: 0
  groups (customer_id) in more than one split: 0
  verdict: no leak of these kinds found

formula: units = rows joined by group and by identical inputs (union-find); units go largest first, ties in seeded random order; each unit goes to the split where Σ_c n_uc·(have_sc − target_sc) is lowest, target_sc = share_s × rows of class c — so classes keep their shares and no unit is cut
source: leakage: Kaufman, Rosset, Perlich & Stitelman (2012), Leakage in data mining: formulation, detection, and avoidance, ACM TKDD 6(4), article 15; Kapoor & Narayanan (2023), Leakage and the reproducibility crisis in machine-learning-based science, Patterns 4(9):100804
limit: shares are as close as whole groups allow (the table shows how close); exact duplicates only: near-duplicates (a resized photo, a reworded sentence) are not found; time order is not checked: for data that will be used on the future, split by time instead; the same seed gives the same split.

$ dataset.py pii tickets.csv --redact tickets-redacted.csv
PII · tickets.csv · 525 rows × 9 columns · values masked; nothing is printed in full

| kind | column | rows | share of rows | example (masked) | first rows |
|---|---|---|---|---|---|
| email | text | 22 | 4.2% | e***@***.com, m***@***.com | 21, 69, 135, 167, 171 and 17 more |
| phone | text | 23 | 4.4% | (***) ***-**14, (***) ***-**11 | 7, 16, 54, 55, 71 and 18 more |
| card number | text | 5 | 1.0% | **** **** **** 1111 | 169, 208, 355, 429, 509 |
| IP address | ip | 525 | 100.0% | 198.***.***.***, 192.***.***.*** | 1, 2, 3, 4, 5 and 520 more |

columns whose names suggest personal data (from the header only):
  customer_name: name
  ip: device or network

wrote tickets-redacted.csv: 575 values replaced by [EMAIL], [PHONE], [CARD], [SSN], [IP] and [DOB]. Names, addresses, dates beside names and free-text details are not touched: read them by eye.

verdict: LIKELY PERSONAL DATA in 2 columns (ip, text) — remove, mask or justify each before training; if the people in it did not agree to this use, stop and run the licence check

formula: patterns, then checks: card numbers must pass the Luhn check; SSN-like numbers must follow the SSA rules (no area 000, 666 or 900-999, no group 00, no serial 0000); IP addresses must parse as IPv4 or IPv6; phone numbers as 3-3-4 digits or + and 10 to 15 digits; dates must be real calendar dates, and count as birth dates when their column or the words before them say so, or as "beside a name" when the row has a name column; k = the size of the smallest group of rows sharing the same combination
source: Luhn (1960), Computer for verifying numbers, US patent 2,950,048 (the Luhn check digit); US Social Security Administration, SSN randomization (in effect 25 June 2011): area 000, 666 and 900-999, group 00 and serial 0000 are never assigned; Python's ipaddress module (IPv4, RFC 791; IPv6, RFC 4291); ITU-T Recommendation E.164 (an international number has at most 15 digits); North American numbers as 3-3-4 digits; Sweeney (2000), Simple demographics often identify people uniquely, Carnegie Mellon University, Data Privacy Working Paper 3; Sweeney (2002), k-anonymity: a model for protecting privacy, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 10(5):557-570
limit: patterns find formats, not people: names in free text, street addresses, faces, voices and rare details (a job title in a small town) are not found; a match can be a test number or a version string; no scan proves a dataset anonymous — Sweeney (2000) estimated that 87% of the US population is unique on ZIP code, sex and date of birth. Whether the law lets you keep it is a question for a privacy lawyer or your data protection officer.

$ dataset.py pii tickets-redacted.csv --examples 0
PII · tickets-redacted.csv · 525 rows × 9 columns · values masked; nothing is printed in full

no emails, phone numbers, card numbers, SSN-like numbers, IP addresses or birth dates found by pattern.

columns whose names suggest personal data (from the header only):
  customer_name: name
  ip: device or network

verdict: NOTHING FOUND BY PATTERN — but the column names above suggest personal data: read them

formula: patterns, then checks: card numbers must pass the Luhn check; SSN-like numbers must follow the SSA rules (no area 000, 666 or 900-999, no group 00, no serial 0000); IP addresses must parse as IPv4 or IPv6; phone numbers as 3-3-4 digits or + and 10 to 15 digits; dates must be real calendar dates, and count as birth dates when their column or the words before them say so, or as "beside a name" when the row has a name column; k = the size of the smallest group of rows sharing the same combination
source: Luhn (1960), Computer for verifying numbers, US patent 2,950,048 (the Luhn check digit); US Social Security Administration, SSN randomization (in effect 25 June 2011): area 000, 666 and 900-999, group 00 and serial 0000 are never assigned; Python's ipaddress module (IPv4, RFC 791; IPv6, RFC 4291); ITU-T Recommendation E.164 (an international number has at most 15 digits); North American numbers as 3-3-4 digits; Sweeney (2000), Simple demographics often identify people uniquely, Carnegie Mellon University, Data Privacy Working Paper 3; Sweeney (2002), k-anonymity: a model for protecting privacy, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 10(5):557-570
limit: patterns find formats, not people: names in free text, street addresses, faces, voices and rare details (a job title in a small town) are not found; a match can be a test number or a version string; no scan proves a dataset anonymous — Sweeney (2000) estimated that 87% of the US population is unique on ZIP code, sex and date of birth. Whether the law lets you keep it is a question for a privacy lawyer or your data protection officer.

$ dataset.py datasheet tickets.csv --name "FAKE support tickets" --label label --agree labels-2.csv --licence CC-BY-4.0 --out DATASHEET.md
DATASHEET · FAKE support tickets → DATASHEET.md
7 sections · 9 lines from the data · 36 open questions for the owner

  Composition · 525 rows × 9 columns in tickets.csv; columns: ticket_id (text), customer_id (text), customer_name (text), created (date), channel (category), ip (text), text (text), response_minutes (integer), label (category).
  Composition · Label column label: billing 192 (36.6%), shipping 160 (30.5%), account 112 (21.3%), other 61 (11.6%).
  Composition · Missing values: response_minutes 17 (3.2%).
  Composition · Exact duplicate rows (not counting the ID column ticket_id): 11 (2.1%).
  Composition · Values outside the 1.5 × IQR fences: response_minutes 27.
  Composition · Personal data found by pattern: IP address in ip (525 rows); phone in text (23 rows); email in text (22 rows); card number in text (5 rows). Say why it is here or remove it.
  Composition · Columns whose names suggest personal data: customer_name (name); ip (device or network).
  Preprocessing, cleaning and labelling · Two labellers on 525 items: 86.5% agreement, Cohen's kappa 0.813 (labels-2.csv).
  Distribution · Licence named: CC-BY-4.0 — Creative Commons Attribution 4.0; credit the source as the licence asks.

formula: the seven sections of a datasheet in the published order; composition prefilled by profile and pii, labelling by agree, distribution by the licence named
source: Gebru, Morgenstern, Vecchione, Vaughan, Wallach, Daume III & Crawford (2021), Datasheets for datasets, Communications of the ACM 64(12):86-92; the model card that points back to it: Mitchell et al. (2019), Model cards for model reporting, Proceedings of FAT* 2019, ACM, 220-229
limit: the questions are this card's own wording of the paper's sections, not its full list; the machine fills only what it can measure, and never answers for the owner.

$ dataset.py licence --licence CC-BY-NC-4.0 --use commercial --changed yes --share no --personal yes --consent unknown --source "a public review set"
LICENCE · provenance, licence and consent

| step | verdict | why |
|---|---|---|
| SOURCE | GO | named: a public review set |
| LICENCE | GO | Creative Commons Attribution-NonCommercial 4.0 — credit the source as the licence asks; no commercial use |
| USE | STOP | the licence forbids commercial use, and yours is commercial |
| USE | CHECK | whether training a model, and selling or sharing it, counts as a use or an adaptation under this licence is a legal question in many places: ask a lawyer before you rely on it |
| PEOPLE | STOP | data about people without recorded consent for this use: do not use it until a privacy lawyer or your data protection officer says how |
| SHARE | GO | not shared |
| RECORDS | CHECK | keep the licence text, the source and any consent records beside the data, in every copy; never strip them |

verdict: STOP — do not use the data this way until each STOP is answered

formula: the checklist order: source → licence → your use against its conditions → the people in it and their consent → sharing → records; any STOP stops
source: the licences as their stewards publish them: Creative Commons 4.0 legal code; Open Data Commons ODbL 1.0, ODC-By 1.0 and PDDL 1.0; the Linux Foundation CDLA-Permissive-2.0; identifiers from the SPDX License List; why the licence file matters more than a site's label: Longpre et al. (2024), A large-scale audit of dataset licensing and attribution in AI, Nature Machine Intelligence 6:975-987
limit: not legal advice; the licence read is the one you name, so check it against the owner's own licence file: Longpre et al. (2024) found hosting sites omitting licences over 70% of the time and miscategorising them over 50%; data protection law (GDPR, CCPA) goes to a privacy lawyer or your data protection officer.

$ dataset.py licence --licence CC-BY-4.0 --use research --changed yes --share no --personal yes --consent yes --source "our help desk, June to September 2026"
LICENCE · provenance, licence and consent

| step | verdict | why |
|---|---|---|
| SOURCE | GO | named: our help desk, June to September 2026 |
| LICENCE | GO | Creative Commons Attribution 4.0 — credit the source as the licence asks |
| USE | GO | a research use is inside the licence |
| USE | CHECK | whether training a model, and selling or sharing it, counts as a use or an adaptation under this licence is a legal question in many places: ask a lawyer before you rely on it |
| PEOPLE | CHECK | people agreed: keep the consent records with the data, and use it only for what they were told |
| SHARE | GO | not shared |
| RECORDS | CHECK | keep the licence text, the source and any consent records beside the data, in every copy; never strip them |

verdict: CHECK — usable on the conditions listed; read each one

formula: the checklist order: source → licence → your use against its conditions → the people in it and their consent → sharing → records; any STOP stops
source: the licences as their stewards publish them: Creative Commons 4.0 legal code; Open Data Commons ODbL 1.0, ODC-By 1.0 and PDDL 1.0; the Linux Foundation CDLA-Permissive-2.0; identifiers from the SPDX License List; why the licence file matters more than a site's label: Longpre et al. (2024), A large-scale audit of dataset licensing and attribution in AI, Nature Machine Intelligence 6:975-987
limit: not legal advice; the licence read is the one you name, so check it against the owner's own licence file: Longpre et al. (2024) found hosting sites omitting licences over 70% of the time and miscategorising them over 50%; data protection law (GDPR, CCPA) goes to a privacy lawyer or your data protection officer.