Choose Big Data Solution

Help users and agents select the right MATLAB tool for large tabular data in file-based formats (CSV, Parquet, delimited text, spreadsheets, MDF). The skill encodes a decision flowchart — recommend one clear path, not a menu of options.

When to Use

• User or agent has large tabular data in file-based formats (CSV, Parquet, delimited text, spreadsheets, MDF)
• User or agent says data is "large", "big", or "huge" (even without a specific size)
• User or agent hits an out-of-memory error with readtable, parquetread, or similar
• User or agent has multiple tabular files to process as a single dataset
• User or agent has multiple tabular files to process independently (per-file)
• User or agent asks how to scale up an existing workflow on tabular file data
• User or agent asks about datastores, tall arrays, or mapreduce
• User or agent needs to build a custom datastore class for a proprietary or non-standard format
• User or agent has large XML, JSON, HTML, or Word document files — readtable supports these formats but the built-in datastores do not. Scaling these requires a custom datastore (see "Formats Without a Built-in Datastore" and "Custom Datastores" sections below)