Unified Data Loader

Unified data loading utilities.

class src.common.data_loader.DataLoader(project_root: str | Path | None = None)[source]

Bases: object

Centralized data loader for all project data.

Eliminates duplicate loading code across modules.

project_root: Root directory of the project.

zone_count: Number of taxi zones (default 263).

slot_count: Number of time slots per day (default 48).

__init__(project_root: str | Path | None = None) → None[source]

Initialize DataLoader.

Parameters:: project_root – Project root directory. If None, auto-detected.

datetime_to_state(dt: datetime) → int[source]

Convert datetime to state index.

Parameters:: dt – Input datetime.
Returns:: State index in [0, 336) where state = weekday * 48 + slot.

load_train_data(train_path: str | Path | None = None, columns: list[str] | None = None) → list[dict[str, Any]][source]

Load cleaned training data.

Parameters:

train_path – Path to train_cleaned.parquet. If None, uses default location.
columns – Columns to load. If None, loads all.

Returns:

List of row dictionaries.

load_travel_time_matrix(travel_time_path: str | Path | None = None) → list[list[float]][source]

Load Dijkstra travel time matrix.

Parameters:: travel_time_path – Path to travel_time_matrix_dijkstra.csv. If None, uses default location.
Returns:: 263x263 matrix where matrix[i][j] is travel time from zone i+1 to zone j+1.
Raises:: ValueError – If matrix dimensions are invalid.

load_zone_statistics(statistics_path: str | Path | None = None) → tuple[list[list[list[float]]], list[list[list[float]]]][source]

Load zone-level demand and fare statistics.

Parameters:: statistics_path – Path to zone_time_statistics.parquet. If None, uses default location.
Returns:: Tuple of (demand, mean_fare) where each is indexed as [weekday][slot][zone_index].

static next_half_hour(value: datetime) → datetime[source]

Round up to the next half-hour boundary.

Parameters:: value – Input datetime.
Returns:: 00 or :30.
Return type:: Datetime rounded up to next

Examples

>>> DataLoader.next_half_hour(datetime(2023, 1, 15, 8, 15))
datetime.datetime(2023, 1, 15, 8, 30)
>>> DataLoader.next_half_hour(datetime(2023, 1, 15, 8, 30))
datetime.datetime(2023, 1, 15, 9, 0)