Skip to content

refac: unified expression filter component across explore, canvas, alerts and reports - #9746

Merged
AdityaHegde merged 45 commits into
mainfrom
refac/unified-filter-component
Sep 7, 2026
Merged

AdityaHegde merged 45 commits into
mainfrom
refac/unified-filter-component

Conversation

@AdityaHegde

@AdityaHegde AdityaHegde commented Jul 23, 2026 •

Copy link
Copy Markdown
Collaborator

We have 3 different filter implementation across explore, canvas, alerts and reports with action code like toggling dimension value duplicated. This PR unifies expression filters components.

The goal of this refactor was also to ensure the state matches UI controls 1-1. Earlier we have V1Expression that doesnt map 1-1, especially with Select vs In-List modes.

  1. ExpressionFilter.svelte acts as the unified filter component that takes a ExpressionFilterManager.
  2. ExpressionFilterManager contains MetricsViewFilterManager per configured metrics views. It also has the full list of DimensionFilterManager/MeasureFilterManager across metrics views, deduped by name. Takes all the filters from MetricsViewFilterManager and creates a sorted list for default filters bar, required first, followed by pinned, followed by dimension and finally measure filters.
  3. JoinerFilterManager is a wrapper per joiner. For existing filter bar it is always an AND joiner. Future PR will support more advanced editing like OR filter and nested AND/OR. Ensures pinned/required filters have an entry. Also handles adding new dimension/measure filter.
  4. DimensionFilterManager encapsulates all actions for a dimension filter. This code was spread out in different places. Along with DimensionFilter.svelte it handles everything for a dimension filter.
  5. Similarly, MeasureFilterManager encapsulates all actions for a measure filter. Along with MeasureFilter.svelte it handles everything for a measure filter.
  6. All of this depends on MetricsViewsProvider. All dimension and measure selectors are come from here.
  7. It also depends on YAMLConfigProvider that provide config that is yaml only and not maintained as a state while rendering dashboard. Currently this has required/pinned filters.
  8. Adds a DashboardConfigProvider for quickly building ExploreDashboardConfigProvider or CanvasDashboardConfigProvider.

For explore,

  1. Adds ExpressionFilterManager to StateManagers.
  2. Replaces the old filter component with the new ExpressionFilter.svelte passing it the ExpressionFilterManager from StateManagers.
  3. Replaces actions/selectors references in leaderbord, dimension table and other places to using ExpressionFilterManager directly.

For canvas,

  1. Adds ExpressionFilterManager to CanvasEntity.
  2. Replaces the old actions with calls to this ExpressionFilterManager.
  3. Still used the old timeAndFilterStore but uses data from parent's ExpressionFilterManager for expression related fields.

There will be a follow up to move time controls as well.

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

@nishantmonu51 nishantmonu51 added Type:Improvement Area:Dashboard Size:XL Very large change: 2,000+ lines labels Aug 4, 2026
@AdityaHegde AdityaHegde changed the title refac: unified filter component across explore, canvas, alerts and reports refac: unified expression filter component across explore, canvas, alerts and reports Aug 6, 2026
@AdityaHegde
AdityaHegde force-pushed the refac/unified-filter-component branch from fb87403 to 0e9eb72 Compare August 10, 2026 14:20
@AdityaHegde
AdityaHegde force-pushed the refac/unified-filter-component branch from 21c97f9 to cc9c8e9 Compare August 11, 2026 12:57
@nishantmonu51

Copy link
Copy Markdown
Collaborator

Reviewed the diff against main with a focus on the new manager classes, the URL/state sync path, and the call sites that were rewritten against them.

1. handleExploreInit takes the update lock without a try/finally

web-common/src/features/dashboards/state-managers/loaders/DashboardStateSync.ts:145

handleExploreInit now sets this.updating = true and expressionFilterManager.updating = true, but unlike its two siblings it does not wrap them in a try/finally. handleURLChange (line 249) does, with an explicit comment that "the finally ensures a throw below cannot leave it stuck", and so does gotoNewState (line 338).

The method is invoked as void this.handleExploreInit(...) from a store subscription and it awaits resolveTimeRanges (a network call) and goto. If either rejects, both locks stay true and initialized stays false for the rest of the session: handleExploreInit and handleURLChange return at their first guard on every later call, gotoNewState no-ops, and syncExpressionFilters (Filters.svelte:403) never writes the filter into the explore store. The dashboard silently stops syncing URL and state.

Before this change the same throw only left initialized false, so the next subscription tick retried.

2. setUrlParams rebuilds every manager on every explore state change

web-common/src/features/dashboards/filters/ExpressionFilterManager.svelte.ts:138

setUrlParams unconditionally rebuilds topLevelJoiner and resets temporaryFilterName, with no short-circuit when the params are unchanged. setParamForMetricsView (line 157) does compare, so the asymmetry looks unintended. DashboardStateSync.gotoNewState:371 calls setUrlParams on every explore state change, including ones unrelated to filters such as sort, time range and the pivot toggle.

Concretely: add an empty filter chip for dimension A from the "+ Filter" menu, then click a leaderboard value for dimension B. B's change round-trips through the explore store into gotoNewState, which calls setUrlParams, and A's pending chip disappears because it has no expression in the URL.

The same mechanism discards staged selections in an open dropdown, since DimensionFilter.svelte:69 derives proxyDimensionManager from the dimensionManager identity and every rebuild replaces that instance.

3. setUrlParams can throw on a malformed gzipped_state

web-common/src/features/dashboards/filters/ExpressionFilterManager.svelte.ts:124, web-common/src/features/dashboards/url-state/compression.ts:32

setUrlParams calls expandCompressedParams unguarded, and decompressUrlParams does a base64 decode plus gunzipSync, both of which throw on malformed input. A truncated or hand-edited shared URL therefore throws out of setUrlParams, and in handleExploreInit that call sits outside any try/finally (see finding 1), which wedges the sync permanently.

mergeFilterParams guards the analogous failure in parseFilterParam (expr-utils.ts:139), so this path is inconsistent with its neighbour.

4. Filters in the URL are dropped when the metrics view specs have not loaded

web-common/src/features/dashboards/filters/ExpressionFilterManager.svelte.ts:139, via JoinerFilterManager.parse:142

Parsing drops any condition whose identifier is absent from metricsViewsProvider.dimensionSpecs, and the parse is imperative rather than derived, so it is never re-run when the specs arrive.

In explore, syncStoreWithSource is called with skipUrlSync = true (Filters.svelte:72-79), so the only callers of setUrlParams are in DashboardStateSync, which is gated on the valid-spec and full-time-range queries rather than on metricsViewsProvider.ready. The specs come from a separate ListResources query and the metrics view name from a separate GetExplore. If ListResources has not resolved when handleExploreInit runs, every filter in the URL is silently discarded and never recovered.

create-alert-utils.ts:79 and scheduled-reports/utils.ts:205 have the sharper version of this: both construct a MetricsViewsProvider and call setUrlParams or setExprForMetricsView on the very next statement, so they only produce the right filters when ListResources happens to be cached already.

5. TDD pinIndex is no longer maintained when filter values change

web-common/src/features/dashboards/time-dimension-details/TimeDimensionDisplay.svelte:151

The deleted toggleMultipleDimensionValueSelections decremented dashboard.tdd.pinIndex when a value at or before the pin was removed, and the deleted clearAllFilters reset it to -1. Neither DimensionFilterManager.toggleValue/removeSelectedValues nor ExpressionFilterManager.clear() does either.

In TDD comparing by dimension, pin the fourth selected value and then unselect the first row: time-dimension-data-store.ts:116 still slices selectedValues.slice(0, pinIndex + 1), so the pin divider lands on the wrong row. Clearing all filters leaves pinIndex above -1 with zero selected values.

6. Nothing emits state-changed, so the subscription in state-managers.ts is dead

web-common/src/features/dashboards/filters/ExpressionFilterManager.svelte.ts:108, web-common/src/features/dashboards/state-managers/state-managers.ts:179

createListener() is an empty stub and "state-changed" is never emitted anywhere in web-common or web-admin, yet state-managers.ts registers a handler for it that calls mergePartialExplorerEntity, and cleanup unsubscribes it. Either the intended sync path was never wired up, or the subscription, the unsub in cleanup, and the event in filter-events.ts:12 should be removed. The doc comment on the event still says it is "diffed and emitted by ExpressionFilterManager.createListener".

7. Leftover debug logging

web-common/src/features/dashboards/state-managers/loaders/DashboardStateSync.ts:60 has console.log("DashboardStateSync constructor"), which fires on every dashboard mount. web-common/src/features/dashboards/filters/test/expression-filters-suite.ts:630 has console.log("Remove...").

8. DimensionFilterManager.clone() loses mode, exclude and input text when there is no expression

web-common/src/features/dashboards/filters/dimension-filters/DimensionFilterManager.svelte.ts:111

clone() reconstructs all three from this.expr alone, so it loses them whenever expr is undefined. DimensionFilter.svelte:69 derives the dropdown's proxy from it.

A chip in Contains mode whose search text has been cleared (commit() sets expr to undefined at line 233), or a required or pinned chip switched to Exclude before any value is picked, reopens in Select mode with Exclude off.

9. setMetricsViewNames leaks time range subscriptions

web-common/src/features/metrics-views/providers/MetricsViewsProvider.svelte.ts:157

setMetricsViewNames replaces the name list but never tears down the timeRangeUnsubs entries for metrics views that dropped out; they are only cleared in cleanup(). A canvas whose YAML stops referencing a metrics view keeps that MetricsViewTimeRange subscription alive for the life of the provider, and the stale timeRangeSummaries and maxQueryTimeRangeMillis entries are never removed.

10. clone() does not forward singleParamFormMv

web-common/src/features/dashboards/filters/ExpressionFilterManager.svelte.ts:110

The clone falls back to the default false, so it silently switches to the f.<mv> param form. No caller passes true today, but isUrlTooLongAfterInListFilter (Filters.svelte:382) measures URL length using the clone, so the length check would measure the wrong URL once the flag is used.

Minor

  • JoinerFilterManager.hasSomeFilter (line 102) and isComplexFilter (line 105) are computed but never read; ExpressionFilterManager.isComplexFilter comes from mergeFilterParams().advanced instead.
  • The added set in parseJoinerExpressionManagers:184 is shared between dimension and measure names, so a dimension and a measure sharing a name would collide during the required and pinned pass.
  • The doc comment on mergeFilterParams says conditions are unioned "in the order the metrics views are given", but it iterates URL param order.

@nishantmonu51 nishantmonu51 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@AdityaHegde
AdityaHegde force-pushed the refac/unified-filter-component branch from 18171de to 6bebff4 Compare September 4, 2026 12:35
@AdityaHegde

AdityaHegde commented Sep 4, 2026 •

Copy link
Copy Markdown
Collaborator Author

Thanks @nishantmonu51 . Fixed all except 1 & 4.

  • The main concern in 1 is that resolveTimeRanges can throw, but it has try-catch around network call and is meant to be robust. I will revisit this in a follow up, try-catch in DashboardStateSync makes it unnecessarily complex.
  • Since ListResources always called this is not an issue. I will do a follow up to improve loading state. There is 2 wrapper components used in all place together, StateManagersProvider & DashboardStateManager.

@AdityaHegde
AdityaHegde merged commit bc0c59c into main Sep 7, 2026
13 of 14 checks passed
@AdityaHegde
AdityaHegde deleted the refac/unified-filter-component branch September 7, 2026 06:23
nishantmonu51 added a commit that referenced this pull request Sep 7, 2026
Resolves conflicts with the unified expression filter refactor (#9746):
adopts the new `where` construction and runes-mode components while
keeping the ephemeral measure request mapping, and points the URL-state
spec at the moved test helpers.

Claude-Session: https://claude.ai/code/session_01NW6ETnqzwL5jdoVCKQPHNf
AdityaHegde pushed a commit that referenced this pull request Sep 24, 2026
Since #9746 migrated DimensionTable.svelte to runes, the column width $effect
assigned estimateColumnSize and then read it back through
estimateColumnSize[0] = manualDimensionColumnWidth. In Svelte 5 an effect that
reads a source it just wrote registers it as a dependency and reschedules
itself, so the first manual resize looped until effect_update_depth_exceeded.
Compute the widths as a $derived instead.
AdityaHegde pushed a commit that referenced this pull request Sep 24, 2026
Since #9746 migrated DimensionTable.svelte to runes, the column width $effect
assigned estimateColumnSize and then read it back through
estimateColumnSize[0] = manualDimensionColumnWidth. In Svelte 5 an effect that
reads a source it just wrote registers it as a dependency and reschedules
itself, so the first manual resize looped until effect_update_depth_exceeded.
Compute the widths as a $derived instead.
nishantmonu51 added a commit that referenced this pull request Sep 25, 2026
…filters (#9951)

`getDimensionFilterWithSearch` returned undefined when the where filter was
undefined, which is now the case for a dashboard with no active filters since
the unified expression filter refactor (#9746). The search text was dropped
and the dimension table query ran without a where clause.

Treat an undefined filter as an empty AND so the search clause is still added.
nishantmonu51 added a commit that referenced this pull request Sep 25, 2026
…filters (#9951)

`getDimensionFilterWithSearch` returned undefined when the where filter was
undefined, which is now the case for a dashboard with no active filters since
the unified expression filter refactor (#9746). The search text was dropped
and the dimension table query ran without a where clause.

Treat an undefined filter as an empty AND so the search clause is still added.
mahdi13 added a commit to inkitt/rill that referenced this pull request Sep 28, 2026
Conflicts:
- FlatTable.svelte, NestedTable.svelte: main moved the row markup into a
  pivotRow snippet (totals row pinning, rilldata#9915). Took main's snippet and
  applied the PIVOT_TOTALS_ROW_ID check there, and updated the comment that
  described the totals row as tanstack row "0".
- pivot-click-to-filter.spec.ts: main rewrote the setup for the unified
  filter manager (rilldata#9746). Took main's version and switched its row ids from
  positional to value-based again.
mahdi13 added a commit to inkitt/rill that referenced this pull request Sep 28, 2026
Conflicts:
- FlatTable.svelte, NestedTable.svelte: main moved the row markup into a
  pivotRow snippet (totals row pinning, rilldata#9915). Took main's snippet and
  applied the PIVOT_TOTALS_ROW_ID check there, and updated the comment that
  described the totals row as tanstack row "0".
- pivot-click-to-filter.spec.ts: main rewrote the setup for the unified
  filter manager (rilldata#9746). Took main's version and switched its row ids from
  positional to value-based again.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area:Dashboard Size:XL Very large change: 2,000+ lines Type:Improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants