Allow positional arguments in distributed queries

[simonmichal]

Changelog category (leave one):

• Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Allow positional arguments in distributed queries

Description

In #62289 positional argument substitution was disabled at remote, however in case of a view that can be expanded only in remote this can lead to NOT_AN_AGGREGATE error. This MR enables positional argument substitutions for views.

Tests

Using the reproducer from #89940 :

:) SELECT *
FROM remote('127.0.0.{1|2|3}', currentDatabase(), experiment_by_platform);

SELECT *
FROM remote('127.0.0.{1|2|3}', currentDatabase(), experiment_by_platform)

Query id: c3175e92-73bc-47c4-99cb-0a4235a0d76c

   ┌─sector──┐
1. │ tech    │
2. │ finance │
3. │ health  │
   └─────────┘

3 rows in set. Elapsed: 0.027 sec.

Making sure fix from #62289 is still working:

:) select 0 as x
from remote('127.0.0.{1|2|3}', system.one)
group by x

SELECT 0 AS x
FROM remote('127.0.0.{1|2|3}', system.one)
GROUP BY x

Query id: 846db696-6be2-470d-825b-6cf3deb90a67

   ┌─x─┐
1. │ 0 │
   └───┘

1 row in set. Elapsed: 0.010 sec.

Version info

• Merged into: 26.5.1.391

[simonmichal]

@devcrafter , @KochetovNicolai - I am happy to make any changes in case I'm missing some context

[devcrafter]

@simonmichal Please add the test

[clickhouse-gh]

Workflow [PR], commit [d6a2af0]

Summary: ✅

---

AI Review

Summary

This PR replaces the previous setting-based workaround with dedicated Context flags so positional arguments are still resolved inside view inner queries on local/remote distributed execution paths, while keeping the existing protection against double-resolution of the outer query. The change is targeted and covered by new stateless tests for both analyzer modes; I did not find additional blocker/major issues in the current diff.

ClickHouse Rules

Item	Status	Notes
Deletion logging	➖
Serialization versioning	➖
Core-area scrutiny	✅
No test removal	✅
Experimental gate	➖
No magic constants	✅
Backward compatibility	✅
SettingsChangesHistory.cpp	➖
PR metadata quality	✅
Safe rollout	✅
Compilation time	✅
No large/binary files	✅

Final Verdict

• Status: ✅ Approve

[simonmichal]

@devcrafter I've added the test, let me know if there's anything else?

regarding the failing BuzzHouse check:

Error:
Logical error: 'Inconsistent AST formatting: the query:
REPLACE TABLE d2.t5 UUID 'c549de1c-69ef-6ecc-3c97-52a893761151' (`c0` Int8 CODEC(NONE, LZ4HC), `c1` FixedString(65) SETTINGS (max_compress_block_size = 1), `c2` DateTime, `c3` QBit(BFloat16, 8), `c4` Array(LowCardinality(Time)) COMMENT 'found', `c5` JSON(max_dynamic_types = 1, max_dynamic_paths = 473, max_dynamic_types = 26) COMMENT '漂亮' CODEC(T64, DoubleDelta(8), AES_128_GCM_SIV) STATISTICS(TDigest, Uniq), `c6` LowCardinality(BFloat16) STATISTICS(TDigest, CountMin, MinMax), `c7` FixedString(99) NULL CODEC(AES_128_GCM_SIV) STATISTICS(CountMin), INDEX i0 c0 TYPE vector_similarity('hnsw', 'L2Distance', 1, 'f64', 4018586, 1265054) GRANULARITY 1, INDEX i1 (c1[-1]) IN (SELECT singleValueOrNull(*) FROM (WITH CAST('49.24.22.79', 'IPv4') AS a0, CAST('68a7e05d-e764-6325-d51c-20f2dbd7c326', 'UUID') AS a1, CAST('304895aa-fc88-cfd4-14f5-01f247997445', 'UUID') AS a2 SELECT CAST('76', 'Int16') FROM (SELECT c0, c1, c2 FROM (WITH cte0d2 AS (SELECT c0, c1, c2, c3, c4 FROM (SELECT CAST(addSeconds(now(), 4), 'DateTime'), CAST('1992-07-07', 'Date') AS a3, a3 AS a4, a4.`叫`.:`Array(JSON)` AS a5, map() OFFSET 0.4853 SETTINGS optimize_time_filter_with_preimage = 1, output_format_parquet_max_dictionary_size = 1073741824, output_format_avro_codec = 'deflate', filesystem_prefetch_step_marks = 0, query_cache_squash_partial_results = 1, input_format_values_accurate_types_of_literals = 1, parallel_replicas_prefer_local_join = 0, query_plan_try_use_vector_search = 1, enable_early_constant_folding = 0, join_output_by_rowlist_perkey_rows_threshold = 0, min_count_to_compile_sort_description = 0, partial_result_on_first_cancel = 1) (c0, c1, c2, c3, c4)), t0d2.number AS a6, cte1d2 AS (SELECT c0, c1, c2 FROM (SELECT NULL, CAST('80.23.28.66', 'IPv4') AS a7, a7 OFFSET 1 SETTINGS enable_global_with_statement = 0, query_plan_join_swap_table = 'auto', database_replicated_always_detach_permanently = 1, move_all_conditions_to_prewhere = 0, input_format_json_read_numbers_as_strings = 1, hdfs_ignore_file_doesnt_exist = 1, max_threads_for_indexes = 13, delta_lake_enable_engine_predicate = 1) (c0, c1, c2)) SELECT NULL, CAST('-885443715538058477499', 'Int256'), t0d2.number AS a8 FROM numbers_mt(498326, 397002) AS t0d2 LEFT JOIN merge('v289') AS t1d2 ON t0d2.number.Int256 = c4 LEFT JOIN `null`(generateRandomStructure(2, 12724589286930684490)) AS t2d2 ON CAST('-24', 'Int8') GROUP BY t0d2.number, t1d2.`c2.size` WINDOW
                    w0 AS (PARTITION BY a6, t1d2.c0, t2d2.c2, CAST('-73786976294838206383', 'Int256')),
                    w1 AS () ORDER BY t1d2.`c2.size` ASC NULLS LAST LIMIT 5 BY 0 SETTINGS correlated_subqueries_default_join_kind = 'left', zstd_window_log_max = 100, allow_dynamic_type_in_join_keys = 1, input_format_arrow_case_insensitive_column_matching = 0, input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference = 0, distributed_plan_max_rows_to_broadcast = 2479, min_insert_block_size_bytes_for_materialized_views = 0, ignore_cold_parts_seconds = 14, use_with_fill_by_sorting_prefix = 1, allow_experimental_query_deduplication = 0, prefer_localhost_replica = 0, query_plan_enable_multithreading_after_window_functions = 0, parallel_replicas_local_plan = 0, least_greatest_legacy_null_behavior = 0, optimize_append_index = 1, output_format_parquet_geometadata = 0, lightweight_delete_mode = 'alter_update', optimize_group_by_function_keys = 1, regexp_dict_flag_case_insensitive = 1, local_filesystem_read_method = 'mmap', optimize_normalize_count_variants = 0, update_insert_deduplication_token_in_dependent_materialized_views = 0, input_format_allow_errors_ratio = 1., alter_move_to_space_execute_async = 1, filesystem_prefetches_limit = 0, optimize_min_inequality_conjunction_chain_length = 9, input_format_csv_allow_whitespace_or_tab_as_delimiter = 1, input_format_parquet_page_filter_push_down = 1, async_insert_threads = 7, alter_move_to_space_execute_async = 0, use_async_executor_for_materialized_views = 0) (c0, c1, c2)) AS t3d1 FULL OUTER JOIN numbers_mt(CAST('2032-07-23 13:44:41', 'DateTime'), 607238, CAST('332041393326771929183', 'UInt256')) AS t4d1 ON (t3d1.c0 = t4d1.number) AND ((a1[_]) = t4d1.number) AND (t4d1.number = a0))) TYPE set(0) GRANULARITY 1, INDEX i2 CAST('072501e1-14e3-8408-0dc1-38dc53980432', 'UUID') AS a9 TYPE bloom_filter GRANULARITY 1, INDEX i3 plus(850, 36171) TYPE minmax GRANULARITY 1, PROJECTION p0 (SELECT c0), PROJECTION p1 (SELECT c5 ORDER BY 1, `c3.16`, `c3.8`, c2, _part_offset._.:`Array(JSON)`) WITH SETTINGS (min_level_for_wide_part = 5, enable_index_granularity_compression = 0, adaptive_write_buffer_initial_size = 0, allow_summing_columns_in_partition_or_order_key = 0, replicated_can_become_leader = 0, compact_parts_merge_max_bytes_to_prefetch_part = 0, notify_newest_block_number = 1, finished_mutations_to_keep = 0, max_merge_delayed_streams_for_parallel_write = 7, allow_part_offset_column_in_projections = 1, max_replicated_mutations_in_queue = 1, always_fetch_merged_part = 0, min_rows_to_fsync_after_merge = 93, remote_fs_zero_copy_path_compatible_mode = 1, shared_merge_tree_outdated_parts_group_size = 8192, materialize_statistics_on_merge = 1, allow_floating_point_partition_key = 0, remove_rolled_back_parts_immediately = 0, merge_selector_heuristic_to_lower_max_parts_to_merge_at_once_exponent = 1, min_age_to_force_merge_on_partition_only = 1, allow_reduce_blocking_parts_task = 0, old_parts_lifetime = 209, shared_merge_tree_partitions_hint_ratio_to_reload_merge_pred_for_mutations = 0., use_minimalistic_part_header_in_zookeeper = 1, exclude_deleted_rows_for_part_size_in_merge = 1, compress_primary_key = 1, vertical_merge_algorithm_min_rows_to_activate = 1855, always_fetch_merged_part = 0, allow_nullable_key = 0, shared_merge_tree_parts_load_batch_size = 97, shared_merge_tree_max_parts_update_leaders_in_total = 0, index_granularity = 8, vertical_merge_algorithm_min_bytes_to_activate = 4, allow_floating_point_partition_key = 1, merge_max_block_size = 512, allow_coalescing_columns_in_partition_or_order_key = 1, allow_suspicious_indices = 1, allow_experimental_reverse_key = 1, nullable_serialization_version = 'allow_sparse', allow_summing_columns_in_partition_or_order_key = 0, allow_vertical_merges_from_compact_to_wide_parts = 1, enable_block_offset_column = 1, min_bytes_for_full_part_storage = 0, min_bytes_for_wide_part = 0, enable_block_number_column = 1, ratio_of_defaults_for_sparse_serialization = 0.0001), PROJECTION p2 (SELECT CAST('885443715538058477626', 'Int128') GROUP BY c7, 1, `c4.size0`, 1, `c3.8`) WITH SETTINGS (max_parts_to_merge_at_once = 1000, merge_selector_base = 8., use_adaptive_write_buffer_for_dynamic_subcolumns = 0, write_marks_for_substreams_in_compact_parts = 0, shared_merge_tree_create_per_replica_metadata_nodes = 0, min_rows_to_fsync_after_merge = 302, table_disk = 1, object_shared_data_buckets_for_wide_part = 4, merge_selector_enable_heuristic_to_remove_small_parts_at_right = 0, remote_fs_zero_copy_path_compatible_mode = 1, disable_detach_partition_for_zero_copy_replication = 1, add_minmax_index_for_string_columns = 0, shared_merge_tree_partitions_hint_ratio_to_reload_merge_pred_for_mutations = 0.786382, remove_empty_parts = 0, replace_long_file_name_to_hash = 1, columns_and_secondary_indices_sizes_lazy_calculation = 1, object_serialization_version = 'v3', merge_selector_enable_heuristic_to_remove_small_parts_at_right = 0, apply_patches_on_merge = 0, search_orphaned_parts_disks = 'any', min_compressed_bytes_to_fsync_after_merge = 1005591, merge_max_dynamic_subcolumns_in_wide_part = 52, merge_selector_window_size = 3708, object_shared_data_buckets_for_wide_part = 10, marks_compression_codec = 'ZSTD', shared_merge_tree_use_metadata_hints_cache = 1, add_minmax_index_for_numeric_columns = 1, replicated_deduplication_window = 0, allow_suspicious_indices = 1, load_existing_rows_count_for_old_parts = 1, max_files_to_modify_in_alter_columns = 98, merge_max_bytes_to_prewarm_cache = 1216210, min_merge_bytes_to_use_direct_io = 0, merge_max_dynamic_subcolumns_in_wide_part = 100)) ENGINE = CollapsingMergeTree(c0) PARTITION BY tuple() ORDER BY tuple() TTL c2 + toIntervalMicrosecond(4) SETTINGS replicated_deduplication_window = 1000, fsync_part_directory = 0, enable_vertical_merge_algorithm = 0, min_rows_for_wide_part = 2549, object_shared_data_serialization_version = 'advanced', enable_block_number_column = 0, shared_merge_tree_max_outdated_parts_to_process_at_once = 524288, shared_merge_tree_use_outdated_parts_compact_format = 0, shared_merge_tree_max_suspicious_broken_parts = 6847, max_bytes_to_merge_at_max_space_in_pool = 6810089, compact_parts_max_granules_to_buffer = 16777216, optimize_row_order = 0, async_insert = 0, max_suspicious_broken_parts = 16777216, marks_compression_codec = 'FPC', min_compress_block_size = 2488282, shared_merge_tree_read_virtual_parts_from_leader = 0, ratio_of_defaults_for_sparse_serialization = 0.03492, deduplicate_merge_projection_mode = 'throw', allow_reduce_blocking_parts_task = 0, shared_merge_tree_activate_coordinated_merges_tasks = 1, cleanup_thread_preferred_points_per_iteration = 1281, min_index_granularity_bytes = 390812, cache_populated_by_fetch = 0, disable_detach_partition_for_zero_copy_replication = 1, refresh_statistics_interval = 0, disable_detach_partition_for_zero_copy_replication = 1, enable_max_bytes_limit_for_min_age_to_force_merge = 1, merge_selector_window_size = 1357, load_existing_rows_count_for_old_parts = 1, shared_merge_tree_use_outdated_parts_compact_format = 0, max_replicated_mutations_in_queue = 43, join_on_disk_max_files_to_merge = 32, filesystem_cache_enable_background_download_for_metadata_files_in_packed_storage = 1, allow_general_join_planning = 0, input_format_custom_skip_trailing_empty_lines = 0, send_progress_in_http_headers = 0, load_marks_asynchronously = 1, allow_push_predicate_ast_for_distributed_subqueries = 0, input_format_json_infer_incomplete_types_as_strings = 1, use_text_index_header_cache = 0, output_format_json_quote_64bit_integers = 0, query_plan_remove_redundant_distinct = 1, cloud_mode_database_engine = 2, max_read_buffer_size = 6207787
cannot parse query back from REPLACE TABLE d2.`t5` UUID 'c549de1c-69ef-6ecc-3c97-52a893761151' (`c0` Int8 CODEC(NONE, LZ4HC), INDEX i0 `c0` TYPE vector_similarity('hnsw', 'L2Distance', 1, 'f64', 4018586, 1265054) GRANULARITY 1, `c1` FixedString(65) SETTINGS(max_compress_block_size = 1), `c2` DateTime, `c3` QBit(BFloat16, 8), INDEX i1 `c1`[-1] == ALL(WITH '49.24.22.79'::IPv4 AS `a0`, '68a7e05d-e764-6325-d51c-20f2dbd7c326'::UUID AS `a1`, '304895aa-fc88-cfd4-14f5-01f247997445'::UUID AS `a2` SELECT 76::Int16 FROM (SELECT `c0`, `c1`, `c2` FROM (WITH cte0d2 AS (SELECT `c0`, `c1`, `c2`, `c3`, `c4` FROM (SELECT addSeconds(now(), 4)::DateTime, '1992-07-07'::Date AS `a3`, `a3` AS `a4`, `a4`.`叫`[] AS `a5`, map() OFFSET 0.4853 SETTINGS optimize_time_filter_with_preimage = 1, output_format_parquet_max_dictionary_size = 1073741824, output_format_avro_codec = 'deflate', filesystem_prefetch_step_marks = 0, query_cache_squash_partial_results = 1, input_format_values_accurate_types_of_literals = 1, parallel_replicas_prefer_local_join = 0, query_plan_try_use_vector_search = 1, enable_early_constant_folding = 0, join_output_by_rowlist_perkey_rows_threshold = 0, min_count_to_compile_sort_description = 0, partial_result_on_first_cancel = 1)(c0, c1, c2, c3, c4)), `t0d2`.`number` AS `a6`, cte1d2 AS (SELECT `c0`, `c1`, `c2` FROM (SELECT NULL, '80.23.28.66'::IPv4 AS `a7`, `a7` OFFSET 1 SETTINGS enable_global_with_statement = 0, query_plan_join_swap_table = 'auto', database_replicated_always_detach_permanently = 1, move_all_conditions_to_prewhere = 0, input_format_json_read_numbers_as_strings = 1, hdfs_ignore_file_doesnt_exist = 1, max_threads_for_indexes = 13, delta_lake_enable_engine_predicate = 1)(c0, c1, c2)) SELECT ALL NULL, -885443715538058477499::Int256, `t0d2`.`number` AS `a8` FROM numbers_mt(498326, 397002) AS t0d2 LEFT JOIN merge('v289') AS t1d2 ON `t0d2`.`number`.`Int256` = `c4` LEFT JOIN `null`(generateRandomStructure(2, 12724589286930684490)) AS t2d2 ON -24::Int8 GROUP BY `t0d2`.`number`, `t1d2`.`c2.size` WINDOW w0 AS (PARTITION BY `a6`, `t1d2`.`c0`, `t2d2`.`c2`, -73786976294838206383::Int256), w1 AS () ORDER BY `t1d2`.`c2.size` NULLS LAST LIMIT 5 BY 0 SETTINGS correlated_subqueries_default_join_kind = 'left', zstd_window_log_max = 100, allow_dynamic_type_in_join_keys = 1, input_format_arrow_case_insensitive_column_matching = 0, input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference = 0, distributed_plan_max_rows_to_broadcast = 2479, min_insert_block_size_bytes_for_materialized_views = 0, ignore_cold_parts_seconds = 14, use_with_fill_by_sorting_prefix = 1, allow_experimental_query_deduplication = 0, prefer_localhost_replica = 0, query_plan_enable_multithreading_after_window_functions = 0, parallel_replicas_local_plan = 0, least_greatest_legacy_null_behavior = 0, optimize_append_index = 1, output_format_parquet_geometadata = 0, lightweight_delete_mode = 'alter_update', optimize_group_by_function_keys = 1, regexp_dict_flag_case_insensitive = 1, local_filesystem_read_method = 'mmap', optimize_normalize_count_variants = 0, update_insert_deduplication_token_in_dependent_materialized_views = 0, input_format_allow_errors_ratio = 1.000000, alter_move_to_space_execute_async = 1, filesystem_prefetches_limit = 0, optimize_min_inequality_conjunction_chain_length = 9, input_format_csv_allow_whitespace_or_tab_as_delimiter = 1, input_format_parquet_page_filter_push_down = 1, async_insert_threads = 7, alter_move_to_space_execute_async = 0, use_async_executor_for_materialized_views = 0)(c0, c1, c2)) AS t3d1 FULL JOIN numbers_mt('2032-07-23 13:44:41'::DateTime, 607238, 332041393326771929183::UInt256) AS t4d1 ON `t3d1`.`c0` = `t4d1`.`number` AND `a1`[`_`] = `t4d1`.`number` AND `t4d1`.`number` = `a0`) TYPE set(0) GRANULARITY 1, `c4` Array(LowCardinality(Time)) COMMENT 'found', INDEX i2 ('072501e1-14e3-8408-0dc1-38dc53980432'::UUID AS `a9`) TYPE bloom_filter GRANULARITY 1, INDEX i3 plus(850, 36171) TYPE minmax, PROJECTION p0 (SELECT `c0`), `c5` JSON(max_dynamic_types=1, max_dynamic_paths=473, max_dynamic_types=26) COMMENT '漂亮' CODEC(T64, DoubleDelta(8), AES_128_GCM_SIV) STATISTICS(TDigest, Uniq), PROJECTION p1 (SELECT `c5` ORDER BY 1, `c3.16`, `c3.8`, `c2`, `_part_offset`.`_`[]) WITH SETTINGS (min_level_for_wide_part = 5, enable_index_granularity_compression = 0, adaptive_write_buffer_initial_size = 0, allow_summing_columns_in_partition_or_order_key = 0, replicated_can_become_leader = 0, compact_parts_merge_max_bytes_to_prefetch_part = 0, notify_newest_block_number = 1, finished_mutations_to_keep = 0, max_merge_delayed_streams_for_parallel_write = 7, allow_part_offset_column_in_projections = 1, max_replicated_mutations_in_queue = 1, always_fetch_merged_part = 0, min_rows_to_fsync_after_merge = 93, remote_fs_zero_copy_path_compatible_mode = 1, shared_merge_tree_outdated_parts_group_size = 8192, materialize_statistics_on_merge = 1, allow_floating_point_partition_key = 0, remove_rolled_back_parts_immediately = 0, merge_selector_heuristic_to_lower_max_parts_to_merge_at_once_exponent = 1, min_age_to_force_merge_on_partition_only = 1, allow_reduce_blocking_parts_task = 0, old_parts_lifetime = 209, shared_merge_tree_partitions_hint_ratio_to_reload_merge_pred_for_mutations = 0.000000, use_minimalistic_part_header_in_zookeeper = 1, exclude_deleted_rows_for_part_size_in_merge = 1, compress_primary_key = 1, vertical_merge_algorithm_min_rows_to_activate = 1855, always_fetch_merged_part = 0, allow_nullable_key = 0, shared_merge_tree_parts_load_batch_size = 97, shared_merge_tree_max_parts_update_leaders_in_total = 0, index_granularity = 8, vertical_merge_algorithm_min_bytes_to_activate = 4, allow_floating_point_partition_key = 1, merge_max_block_size = 512, allow_coalescing_columns_in_partition_or_order_key = 1, allow_suspicious_indices = 1, allow_experimental_reverse_key = 1, nullable_serialization_version = 'allow_sparse', allow_summing_columns_in_partition_or_order_key = 0, allow_vertical_merges_from_compact_to_wide_parts = 1, enable_block_offset_column = 1, min_bytes_for_full_part_storage = 0, min_bytes_for_wide_part = 0, enable_block_number_column = 1, ratio_of_defaults_for_sparse_serialization = 0.0001), `c6` LowCardinality(BFloat16) STATISTICS(TDigest, CountMin, MinMax), `c7` FixedString(99) NULL CODEC(AES_128_GCM_SIV) STATISTICS(CountMin), PROJECTION p2 (SELECT 885443715538058477626::Int128 GROUP BY `c7`, 1, `c4.size0`, 1, `c3.8`) WITH SETTINGS (max_parts_to_merge_at_once = 1000, merge_selector_base = 8.000000, use_adaptive_write_buffer_for_dynamic_subcolumns = 0, write_marks_for_substreams_in_compact_parts = 0, shared_merge_tree_create_per_replica_metadata_nodes = 0, min_rows_to_fsync_after_merge = 302, table_disk = 1, object_shared_data_buckets_for_wide_part = 4, merge_selector_enable_heuristic_to_remove_small_parts_at_right = 0, remote_fs_zero_copy_path_compatible_mode = 1, disable_detach_partition_for_zero_copy_replication = 1, add_minmax_index_for_string_columns = 0, shared_merge_tree_partitions_hint_ratio_to_reload_merge_pred_for_mutations = 0.786382, remove_empty_parts = 0, replace_long_file_name_to_hash = 1, columns_and_secondary_indices_sizes_lazy_calculation = 1, object_serialization_version = 'v3', merge_selector_enable_heuristic_to_remove_small_parts_at_right = 0, apply_patches_on_merge = 0, search_orphaned_parts_disks = 'any', min_compressed_bytes_to_fsync_after_merge = 1005591, merge_max_dynamic_subcolumns_in_wide_part = 52, merge_selector_window_size = 3708, object_shared_data_buckets_for_wide_part = 10, marks_compression_codec = 'ZSTD', shared_merge_tree_use_metadata_hints_cache = 1, add_minmax_index_for_numeric_columns = 1, replicated_deduplication_window = 0, allow_suspicious_indices = 1, load_existing_rows_count_for_old_parts = 1, max_files_to_modify_in_alter_columns = 98, merge_max_bytes_to_prewarm_cache = 1216210, min_merge_bytes_to_use_direct_io = 0, merge_max_dynamic_subcolumns_in_wide_part = 100)) ENGINE = CollapsingMergeTree(`c0`) ORDER BY tuple() PARTITION BY tuple() TTL `c2` + INTERVAL (4) MICROSECOND SETTINGS replicated_deduplication_window = 1000, fsync_part_directory = 0, enable_vertical_merge_algorithm = 0, min_rows_for_wide_part = 2549, object_shared_data_serialization_version = 'advanced', enable_block_number_column = 0, shared_merge_tree_max_outdated_parts_to_process_at_once = 524288, shared_merge_tree_use_outdated_parts_compact_format = 0, shared_merge_tree_max_suspicious_broken_parts = 6847, max_bytes_to_merge_at_max_space_in_pool = 6810089, compact_parts_max_granules_to_buffer = 16777216, optimize_row_order = 0, async_insert = 0, max_suspicious_broken_parts = 16777216, marks_compression_codec = 'FPC', min_compress_block_size = 2488282, shared_merge_tree_read_virtual_parts_from_leader = 0, ratio_of_defaults_for_sparse_serialization = 0.034920, deduplicate_merge_projection_mode = 'throw', allow_reduce_blocking_parts_task = 0, shared_merge_tree_activate_coordinated_merges_tasks = 1, cleanup_thread_preferred_points_per_iteration = 1281, min_index_granularity_bytes = 390812, cache_populated_by_fetch = 0, disable_detach_partition_for_zero_copy_replication = 1, refresh_statistics_interval = 0, disable_detach_partition_for_zero_copy_replication = 1, enable_max_bytes_limit_for_min_age_to_force_merge = 1, merge_selector_window_size = 1357, load_existing_rows_count_for_old_parts = 1, shared_merge_tree_use_outdated_parts_compact_format = 0, max_replicated_mutations_in_queue = 43, join_on_disk_max_files_to_merge = 32, filesystem_cache_enable_background_download_for_metadata_files_in_packed_storage = 1, allow_general_join_planning = 0, input_format_custom_skip_trailing_empty_lines = 0, send_progress_in_http_headers = 0, load_marks_asynchronously = 1, allow_push_predicate_ast_for_distributed_subqueries = 0, input_format_json_infer_incomplete_types_as_strings = 1, use_text_index_header_cache = 0, output_format_json_quote_64bit_integers = 0, query_plan_remove_redundant_distinct = 1, cloud_mode_database_engine = 2, max_read_buffer_size = 6207787;'.

it seems to me it's unrelated

[alexey-milovidov]

This is good, looks almost ready for merge.

[simonmichal]

@devcrafter , @alexey-milovidov : just a friendly ping to see if there's anything else that needs to be adjusted?

[alexey-milovidov]

@simonmichal, the test failed.

[simonmichal]

@alexey-milovidov I beg your pardon,

apparently I've been slopy, I've generated the reference with

$ build/programs/clickhouse-client < tests/queries/0_stateless/03802_positional_arg_in_view_distributed_query.sql  > tests/queries/0_stateless/03802_positional_arg_in_view_distributed_query.reference

but apparently I didn't run

$ ./tests/clickhouse-test 03802_positional_arg_in_view_distributed_query

for some reason running:

$ build/programs/clickhouse-client < tests/queries/0_stateless/03802_positional_arg_in_view_distributed_query.sql

always works (at least on my local setup)

however, running it through the testsuit fails

$ ./tests/clickhouse-test 03802_positional_arg_in_view_distributed_query

Received exception from server (version 26.2.1):
Code: 215. DB::Exception: Received from localhost:9000. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Column 'test_mv5fd6wx.test_table.str' is not under aggregate function and not in GROUP BY keys. In query SELECT str FROM test_mv5fd6wx.test_table GROUP BY 1. (NOT_AN_AGGREGATE)
(query: -- Simulate distributed query to "remote" nodes (points back to localhost or multiple addresses)
select str
from remote('127.0.0.{1|2|3}', currentDatabase(), test_view)
order by str;)
, result:

a
b
c

stdout:
a
b
c

I will debug it on the weekend

[simonmichal]

@alexey-milovidov , @devcrafter sorry for the delay!

It turns out that apparently if the test is execute via ./tests/clickhouse-test the query gets marked as SECONDARY_QUERY:

572     void QueryAnalyzer::replaceNodesWithPositionalArguments(QueryTreeNodePtr & node_list, const QueryTreeNodes & projection_nodes, IdentifierResolveScope & scope)
573     {
574         const auto & settings = scope.context->getSettingsRef();
575         if (!settings[Setting::enable_positional_arguments] || scope.context->getClientInfo().query_kind != ClientInfo::QueryKind::INITIAL_QUERY)
576             return;
577
(gdb) p scope.context->getClientInfo().query_kind
$11 = DB::ClientInfo::QueryKind::SECONDARY_QUERY

I assumed that a view should be always treated as INITIAL_QUERY upon expanding - this should fix the tests.

Let me know if this seems OK?

[devcrafter]

It turns out that apparently if the test is execute via ./tests/clickhouse-test the query gets marked as SECONDARY_QUERY:

572     void QueryAnalyzer::replaceNodesWithPositionalArguments(QueryTreeNodePtr & node_list, const QueryTreeNodes & projection_nodes, IdentifierResolveScope & scope)
573     {
574         const auto & settings = scope.context->getSettingsRef();
575         if (!settings[Setting::enable_positional_arguments] || scope.context->getClientInfo().query_kind != ClientInfo::QueryKind::INITIAL_QUERY)
576             return;
577
(gdb) p scope.context->getClientInfo().query_kind
$11 = DB::ClientInfo::QueryKind::SECONDARY_QUERY

I assumed that a view should be always treated as INITIAL_QUERY upon expanding - this should fix the tests.

Let me know if this seems OK?

(1) QueryKind::INITIAL_QUERY is related to query execution on initiator (the node which accept initial query from client), (2) QueryKind::SECONDARY_QUERY when the query or part of the initial query is executed on remote node. For (2) the result of its execution is sent back to the initiator, and there the query execution is finished (the result is sent back to client)

[simonmichal]

@devcrafter thanks for the clarification!

regarding the failing test, what's the nature of Performance Comparison (amd_release, master_head, 6/6)?

is it one of the tests in tests/performance ?

if marking it as INITIAL_QUERY has negative performance impact I can add a new flag to the context, i.e. is_view_expansion and use it to determine if replaceNodesWithPositionalArguments shouldn't be short-circuited

let me know your thoughts!

[simonmichal]

@devcrafter

(1) QueryKind::INITIAL_QUERY is related to query execution on initiator (the node which accept initial query from client), (2) QueryKind::SECONDARY_QUERY when the query or part of the initial query is executed on remote node. For (2) the result of its execution is sent back to the initiator, and there the query execution is finished (the result is sent back to client)

I am wondering, in the test environment do both the initiator and the executor run the MR? I am bit puzzlet why the view was not expanded properly on the initiator where the flag is set to QueryKind::INITIAL_QUERY.

[simonmichal]

@devcrafter : just a friendly ping, I was wondering if there are suggestions on your side?

[simonmichal]

@devcrafter @alexey-milovidov : I was just wondering if there's any advice on your side on how to proceed?

• regarding the initial problem with the 03802_positional_arg_in_view_distributed_query test failing and @devcrafter comment:

(1) QueryKind::INITIAL_QUERY is related to query execution on initiator (the node which accept initial query from client), (2) QueryKind::SECONDARY_QUERY when the query or part of the initial query is executed on remote node. For (2) the result of its execution is sent back to the initiator, and there the query execution is finished (the result is sent back to client)

I am wondering, if in the test environment both: the initiator and the executor run the MR (that is being tested)? I am bit puzzlet why the view was not expanded properly on the initiator where the flag is set to QueryKind::INITIAL_QUERY - hence my question - it crossed my mind that it might be that only the executor runs the MR and in this case the failure would be a false positive

• regarding current failure in the CI: Performance Comparison (amd_release, master_head, 6/6) - I don't have enough intuition to judge if this could be caused by setting ClientInfo::QueryKind::INITIAL_QUERY in view context - could you please advice?

Please let me know your thoughts!

[devcrafter]

@devcrafter @alexey-milovidov : I was just wondering if there's any advice on your side on how to proceed?

* regarding the initial problem with the `03802_positional_arg_in_view_distributed_query` test failing and @devcrafter comment:

(1) QueryKind::INITIAL_QUERY is related to query execution on initiator (the node which accept initial query from client), (2) QueryKind::SECONDARY_QUERY when the query or part of the initial query is executed on remote node. For (2) the result of its execution is sent back to the initiator, and there the query execution is finished (the result is sent back to client)

I am wondering, if in the test environment both: the initiator and the executor run the MR (that is being tested)? I am bit puzzlet why the view was not expanded properly on the initiator where the flag is set to QueryKind::INITIAL_QUERY - hence my question - it crossed my mind that it might be that only the executor runs the MR and in this case the failure would be a false positive

* regarding current failure in the CI: `Performance Comparison (amd_release, master_head, 6/6)` - I don't have enough intuition to judge if this could be caused by setting `ClientInfo::QueryKind::INITIAL_QUERY` in view context - could you please advice?

Please let me know your thoughts!

Sorry for the delay, I'll check it shortly

[simonmichal]

@devcrafter: it’s ready for another review:

• The CI pipeline failures are all due to 04122, which seems unrelated.
• The current approach uses two new flags (is_view_inner_query and positional_arguments_already_resolved) in the context to decide whether positional arguments should be resolved.
• I kept the commit history, but if that looks too messy, let me know—I can squash it.

[clickhouse-gh]

LLVM Coverage Report

Metric	Baseline	Current	Δ
Lines	84.10%	84.10%	+0.00%
Functions	91.10%	91.10%	+0.00%
Branches	76.60%	76.60%	+0.00%

Changed lines: 100.00% (53/53) · Uncovered code

Full report · Diff report

[simonmichal]

@devcrafter and @alexey-milovidov: freshly rebased, all green, ready for review ;-) I hope you guys didn't give up on this one — it took me some time, but I think it's now in a good state :-)

A long time has passed.