Update Results for DataFusion version to 54.0.0 by alamb · Pull Request #948 · ClickHouse/ClickBench

alamb · 2026-06-18T19:25:43Z

Part of Update ClickBench benchmarks with DataFusion 54.0.0 (when released) apache/datafusion#21505

This updates ClickBench with results for DataFusion 54 - https://datafusion.apache.org/blog/output/2026/06/12/datafusion-54.0.0/

Contains updates for:

`datafusion`

c6a.xlarge — x86_64 (AMD) — 4 vCPU
c6a.2xlarge — x86_64 (AMD) — 8 vCPU
c6a.4xlarge — x86_64 (AMD) — 16 vCPU — the canonical/reference size
c6a.metal — x86_64 (AMD) — bare metal
c7a.metal-48xl — x86_64 (AMD) — bare metal, 192 vCPU
c8g.4xlarge — arm64 (Graviton4) — 16 vCPU
c8g.metal-48xl — arm64 (Graviton4) — bare metal, 192 vCPU

`datafusion-partitioned`

c6a.xlarge — x86_64 (AMD) — 4 vCPU
c6a.2xlarge — x86_64 (AMD) — 8 vCPU
c6a.4xlarge — x86_64 (AMD) — 16 vCPU — the canonical/reference size
c6a.metal — x86_64 (AMD) — bare metal
c7a.metal-48xl — x86_64 (AMD) — bare metal, 192 vCPU
c8g.4xlarge — arm64 (Graviton4) — 16 vCPU
c8g.metal-48xl — arm64 (Graviton4) — bare metal, 192 vCPU

I generated these numbers using aws_clickbench.txt, with the help of Claude Code:

alamb · 2026-06-19T11:17:50Z

    fi
    cd arrow-datafusion
-    git checkout 53.1.0
+    git checkout 54.0.0


here is code change 1

alamb · 2026-06-19T11:17:55Z

    fi
    cd arrow-datafusion
-    git checkout 53.1.0
+    git checkout 54.0.0


here is code change 2

alamb · 2026-06-19T12:51:32Z

This PR is now ready for review

alamb · 2026-06-19T12:52:32Z

-,{"system":"DataFusion (Parquet, partitioned)","date":"2026-05-11","machine":"c7a.metal-48xl","cluster_size":1,"proprietary":"no","hardware":"cpu","tuned":"no","tags":["Rust","column-oriented","embedded","stateless"],"load_time":54,"data_size":14737666736,"concurrent_qps":7.017,"concurrent_error_ratio":0,"result":[[0.018,0.003,0.002],[0.07,0.039,0.042],[0.128,0.067,0.06],[0.41,0.077,0.077],[0.495,0.212,0.203],[0.86,0.19,0.22],[0.066,0.037,0.038],[0.092,0.047,0.05],[0.765,0.269,0.26],[1.485,0.349,0.341],[0.511,0.115,0.132],[0.586,0.121,0.13],[0.862,0.222,0.209],[2.237,0.366,0.361],[0.942,0.227,0.209],[0.513,0.231,0.225],[2.191,0.367,0.371],[2.167,0.371,0.352],[4.253,0.731,0.668],[0.189,0.062,0.071],[9.496,0.329,0.335],[11.162,0.385,0.409],[21.84,0.51,0.465],[49.342,0.629,0.672],[1.786,0.055,0.072],[0.799,0.094,0.088],[1.797,0.116,0.118],[9.714,0.301,0.304],[8.472,1.74,1.669],[0.166,0.13,0.128],[2.893,0.252,0.255],[6.435,0.351,0.329],[4.719,1.112,1.132],[9.622,0.542,0.524],[9.618,0.506,0.588],[0.35,0.288,0.263],[0.223,0.124,0.125],[0.162,0.063,0.063],[0.216,0.076,0.065],[0.391,0.227,0.233],[0.115,0.042,0.045],[0.114,0.033,0.039],[0.101,0.05,0.054]],"source":"datafusion-partitioned/results/20260511/c7a.metal-48xl.json"}
-,{"system":"DataFusion (Parquet, partitioned)","date":"2026-05-11","machine":"c8g.4xlarge","cluster_size":1,"proprietary":"no","hardware":"cpu","tuned":"no","tags":["Rust","column-oriented","embedded","stateless"],"load_time":18,"data_size":14737666736,"concurrent_qps":0.095,"concurrent_error_ratio":0,"result":[[0.015,0.002,0.002],[0.075,0.014,0.012],[0.206,0.051,0.054],[0.511,0.046,0.042],[0.556,0.256,0.25],[0.965,0.373,0.357],[0.044,0.005,0.005],[0.083,0.015,0.015],[0.84,0.331,0.34],[1.613,0.552,0.576],[0.594,0.094,0.092],[0.687,0.111,0.099],[0.978,0.371,0.361],[2.425,0.485,0.467],[1.077,0.353,0.381],[0.575,0.277,0.277],[2.358,0.562,0.561],[2.343,0.547,0.549],[4.601,1.092,1.102],[0.258,0.039,0.038],[9.731,0.585,0.586],[11.477,0.623,0.636],[22.246,1.006,1.001],[53.834,2.417,2.441],[0.249,0.075,0.076],[0.856,0.133,0.147],[0.257,0.078,0.075],[10.003,0.725,0.704],[9.172,6.667,6.677],[0.447,0.35,0.347],[3.054,0.333,0.317],[6.802,0.324,0.358],[4.831,0.904,0.894],[9.909,1.208,1.194],[9.914,1.202,1.201],[0.655,0.477,0.497],[0.291,0.129,0.131],[0.137,0.049,0.049],[0.237,0.083,0.083],[0.495,0.241,0.241],[0.123,0.021,0.023],[0.105,0.02,0.02],[0.104,0.02,0.021]],"source":"datafusion-partitioned/results/20260511/c8g.4xlarge.json"}
-,{"system":"DataFusion (Parquet, partitioned)","date":"2026-05-11","machine":"c8g.metal-48xl","cluster_size":1,"proprietary":"no","hardware":"cpu","tuned":"no","tags":["Rust","column-oriented","embedded","stateless"],"load_time":45,"data_size":14737666736,"concurrent_qps":6.498,"concurrent_error_ratio":0,"result":[[0.013,0.002,0.002],[0.058,0.018,0.019],[0.218,0.041,0.043],[0.404,0.045,0.043],[0.445,0.172,0.161],[0.826,0.168,0.173],[0.067,0.017,0.017],[0.082,0.027,0.029],[0.699,0.215,0.221],[1.406,0.273,0.262],[0.498,0.099,0.092],[0.569,0.105,0.103],[0.847,0.171,0.175],[2.2,0.295,0.332],[0.871,0.157,0.163],[0.453,0.203,0.221],[2.123,0.345,0.318],[2.125,0.322,0.294],[4.292,0.588,0.622],[0.31,0.043,0.052],[9.495,0.282,0.245],[11.162,0.249,0.291],[21.822,0.503,0.504],[49.13,0.694,0.819],[1.964,0.081,0.089],[0.788,0.062,0.066],[1.952,0.093,0.094],[9.734,0.304,0.239],[8.554,1.943,1.981],[0.196,0.144,0.146],[2.86,0.184,0.204],[6.419,0.303,0.292],[4.752,0.873,0.887],[9.598,0.504,0.508],[9.564,0.553,0.532],[0.456,0.195,0.186],[0.21,0.097,0.098],[0.148,0.054,0.056],[0.225,0.062,0.059],[0.355,0.184,0.184],[0.103,0.034,0.033],[0.098,0.029,0.03],[0.094,0.029,0.029]],"source":"datafusion-partitioned/results/20260511/c8g.metal-48xl.json"}
+,{"system":"DataFusion (Parquet, partitioned)","date":"2026-06-18","machine":"c6a.2xlarge","cluster_size":1,"proprietary":"no","hardware":"cpu","tuned":"no","tags":["Rust","column-oriented","embedded","stateless"],"load_time":8,"data_size":14737666736,"concurrent_qps":null,"concurrent_error_ratio":null,"result":[[0.07,0.002,0.002],[0.131,0.036,0.036],[0.561,0.109,0.107],[0.86,0.109,0.105],[1.906,0.912,0.919],[1.756,0.826,0.815],[0.073,0.002,0.002],[0.152,0.037,0.038],[1.734,1.135,1.14],[1.594,1.153,1.138],[0.608,0.227,0.226],[0.642,0.255,0.258],[1.142,0.793,0.789],[2.624,1.224,1.118],[1.146,0.794,0.793],[1.364,1.009,0.995],[3.019,1.837,1.835],[2.991,1.83,1.829],[6.053,3.689,3.683],[0.264,0.093,0.094],[9.971,1.084,1.081],[11.396,1.339,1.336],[22.245,2.421,2.432],[54.84,44.429,42.869],[0.548,0.089,0.091],[0.876,0.255,0.259],[0.24,0.093,0.094],[9.852,1.458,1.45],[8.633,6.539,6.511],[0.215,0.099,0.098],[3.238,0.839,0.842],[6.968,0.864,0.856],[4.969,3.133,3.131],[10.813,3.832,3.806],[10.785,3.754,3.759],[1.062,0.862,0.861],[0.235,0.08,0.079],[0.165,0.042,0.041],[0.201,0.054,0.046],[0.331,0.152,0.158],[0.141,0.021,0.023],[0.129,0.023,0.023],[0.132,0.018,0.02]],"source":"datafusion-partitioned/results/20260618/c6a.2xlarge.json"}


I updated this file using the scripts fix here

Fix generate-results.sh: avoid "Argument list too long" #949

ANd then I manually reverted unrelated changes (alloydb, a bunch of ClickHouse cloud entries)

…on_54 # Conflicts: # data.generated.js

alamb · 2026-06-29T17:40:15Z

Hi @alexey-milovidov and @george-larionov - could you please review this new contribution (or feel free to just take the code change and rerun the results with your own infrastructure)

…on_54

alamb · 2026-06-29T19:31:12Z

Thank you @rschu1ze 🙏

alamb changed the title ~~Update DataFusion version to 54.0.0~~ Update Results for DataFusion version to 54.0.0 Jun 19, 2026

alamb force-pushed the alamb/update_datafusion_54 branch from dc49fcd to b716125 Compare June 19, 2026 11:17

alamb commented Jun 19, 2026

View reviewed changes

This was referenced Jun 19, 2026

Fix generate-results.sh: avoid "Argument list too long" #949

Open

Update ClickBench benchmarks with DataFusion 54.0.0 (when released) apache/datafusion#21505

Open

alamb force-pushed the alamb/update_datafusion_54 branch from b716125 to 2bffcd1 Compare June 19, 2026 12:37

Update DataFusion version to 54.0.0

200098a

alamb force-pushed the alamb/update_datafusion_54 branch from 2bffcd1 to 200098a Compare June 19, 2026 12:47

alamb marked this pull request as ready for review June 19, 2026 12:51

alamb commented Jun 19, 2026

View reviewed changes

alamb mentioned this pull request Jun 20, 2026

fix(datafusion): Update docs, skip concurrent_qps, Skip re-downloading partitioned files #943

Merged

alamb added 2 commits June 29, 2026 13:01

Merge remote-tracking branch 'origin/main' into alamb/update_datafusi…

13cc6fe

…on_54 # Conflicts: # data.generated.js

Revert generate-results script change

8259574

rschu1ze self-assigned this Jun 29, 2026

rschu1ze added 4 commits June 29, 2026 18:21

Remove manually removed JS file

8caa16a

Update README

acbde08

Merge remote-tracking branch 'origin/main' into alamb/update_datafusi…

87b36b8

…on_54

Remove duplicate measurements

14c8d2e

rschu1ze force-pushed the alamb/update_datafusion_54 branch from 0d62563 to 14c8d2e Compare June 29, 2026 19:03

rschu1ze merged commit 9f5d9bc into ClickHouse:main Jun 29, 2026

rschu1ze mentioned this pull request Jun 29, 2026

Update results for Datafusion 54.0.0 #959

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update Results for DataFusion version to 54.0.0#948

Update Results for DataFusion version to 54.0.0#948
rschu1ze merged 7 commits into
ClickHouse:mainfrom
alamb:alamb/update_datafusion_54

alamb commented Jun 18, 2026 •

edited by rschu1ze

Loading

Uh oh!

alamb Jun 19, 2026

Uh oh!

alamb Jun 19, 2026

Uh oh!

alamb commented Jun 19, 2026

Uh oh!

alamb Jun 19, 2026

Uh oh!

alamb commented Jun 29, 2026

Uh oh!

alamb commented Jun 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

alamb commented Jun 18, 2026 • edited by rschu1ze Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

datafusion

datafusion-partitioned

Uh oh!

alamb Jun 19, 2026

Choose a reason for hiding this comment

Uh oh!

alamb Jun 19, 2026

Choose a reason for hiding this comment

Uh oh!

alamb commented Jun 19, 2026

Uh oh!

alamb Jun 19, 2026

Choose a reason for hiding this comment

Uh oh!

alamb commented Jun 29, 2026

Uh oh!

alamb commented Jun 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alamb commented Jun 18, 2026 •

edited by rschu1ze

Loading

`datafusion`

`datafusion-partitioned`