Hey @alexey-milovidov, @rschu1ze,
A colleague of mine at Firebolt a couple of days ago activated "Machine: All" on ClickBench and was surprised to see "Firebolt (Parquet)" on a t3a.small (a tiny instance type) at the top 😅.
Looking at the runs https://github.com/ClickHouse/ClickBench/blob/main/firebolt-parquet/results/20260511/t3a.small.json these look far too fast for many of the queries.
We think this may be a mistake when automatically collecting the runs on various machines (thanks for doing this btw, very cool 💥!). Our guess is that the queries error out and that the error handling here: https://github.com/ClickHouse/ClickBench/blob/main/firebolt-parquet/query#L22 somehow doesn't catch this.
Could you please double-check and if you agree, remove the "too good to be true" results? Thanks!
Hey @alexey-milovidov, @rschu1ze,
A colleague of mine at Firebolt a couple of days ago activated "Machine: All" on ClickBench and was surprised to see "Firebolt (Parquet)" on a
t3a.small(a tiny instance type) at the top 😅.Looking at the runs https://github.com/ClickHouse/ClickBench/blob/main/firebolt-parquet/results/20260511/t3a.small.json these look far too fast for many of the queries.
We think this may be a mistake when automatically collecting the runs on various machines (thanks for doing this btw, very cool 💥!). Our guess is that the queries error out and that the error handling here: https://github.com/ClickHouse/ClickBench/blob/main/firebolt-parquet/query#L22 somehow doesn't catch this.
Could you please double-check and if you agree, remove the "too good to be true" results? Thanks!