Evaluation Dimensions
Data Extraction
Extract numerical values, percentages, and data points from charts accurately.
Chart Recognition
Identify chart types, subtypes, and structural elements correctly.
Trend Analysis
Detect patterns, trends, inflection points, and seasonality.
Comparison
Compare values, identify extremes, and determine rankings.
Visual Reasoning
Multi-step inference, causal reasoning, and insight generation.
OCR Accuracy
Read labels, titles, legends, and annotations accurately.
Mathematical
Perform calculations like sums, averages, and growth rates.
Anomaly Detection
Identify outliers, missing data, and unusual patterns.
Benchmark Suites
CharXiv
Academic and scientific charts from research papers. Tests understanding of complex data visualizations used in scholarly work.
50 tests
Scientific focus
ChartQAPro
Professional and business charts. Tests real-world chart understanding for enterprise applications and decision-making.
50 tests
Business focus
ChartMuseum
Diverse collection spanning all chart types. Tests breadth of understanding across various visualization styles.
50 tests
Comprehensive
Chart Types Covered
Difficulty Distribution
Basic
40
Simple single-step tasks
Intermediate
45
Multi-element reasoning
Advanced
36
Complex multi-step
Expert
29
Domain expertise required