Parquet Converter

Business Case

Problem Statement

Data storage and processing challenges:

• Large CSV files are slow to process
• Inefficient storage of typed data
• Column-oriented queries are slow
• Incompatible with modern data platforms

Solution

Convert construction data to Parquet format for efficient columnar storage, faster queries, and compatibility with data lakehouses.

Technical Implementation

import pandas as pd
from typing import Dict, Any, List, Optional, Union
from dataclasses import dataclass, field