5. Cleaning and Preparing Datasets Free Online FlashCards

What is a dataset?

A dataset is an organized collection of observations, often stored as a spreadsheet, CSV file, database table, or specialized format such as Parquet.

How do a field and a record differ?

A field, or column, describes one characteristic; a record, or row, contains the values associated with one case, event, or entity.

What does a data dictionary define?

A data dictionary or schema records field meanings, expected types, units, permitted values, coding conventions, table relationships, intended use, and limitations.

What makes an identifier a primary key?

A primary key is an identifier with a unique value for each valid record. It distinguishes records but should not be treated as a meaningful measurement or ranking.

Why preserve the raw dataset?

Preserving raw data keeps an unchanged copy of the source file, allowing transformations to be traced, repeated, checked, or reversed.

Why must missing values be distinguished from zero?

A blank or missing value does not automatically mean zero. Zero may represent a measured quantity, whereas a blank may mean unavailable, inapplicable, withheld, or unrecorded.

What is imputation?

Imputation estimates a replacement value using a justified method, such as a group median or interpolation. The replacement remains an estimate, not an observed fact.

What is the purpose of standardization?

Standardization gives equivalent values one consistent representation, such as approved category codes or ISO dates, without erasing distinctions that carry meaning.

What problem can duplicate records cause?

A duplicate is an unnecessary repeated record, either exactly or through a repeated identifier. Duplicates can cause observations to be counted more than once.

What is an outlier?

An outlier is a value unusually far from most others. It may be an error, a unit mistake, a valid rare observation, or evidence of changed conditions.

How should an outlier be handled?

Outliers should be flagged and investigated against source records, domain limits, and context before removal. Valid rare observations may need to be retained.

What does a range check test?

A range check tests whether a value lies within a reasonable interval, such as 0 to 100 for a percentage or 0 to 1 for a probability.