Skip to content

[$500] Don't change the order of user-entered search syntax #69394

Closed

Description

@JmillsExpensify

Let's not change the order of a user-entered query, as it's confusing and unexpected. Here's a recording, notice how the group-by syntax jumps around? Let's make sure we're not doing that.

Image

cc @s77rt @ikevin127

Upwork Automation - Do Not Edit
  • Upwork Job URL: https://www.upwork.com/jobs/~021960818749737255277
  • Upwork Job ID: 1960818749737255277
  • Last Price Increase: 2025-12-02
  • Automatic offers:
    • ShridharGoel | Contributor | 108861845
Issue OwnerCurrent Issue Owner: @

Activity

  1. added
    BugSomething is broken. Auto assigns a BugZero manager.
    on Aug 27, 2025
  2. melvin-bot commented on Aug 27, 2025

    @melvin-bot

    Current assignee @JmillsExpensify is eligible for the Bug assigner, not assigning anyone new.

  3. added
    ExternalAdded to denote the issue can be worked on by a contributor
    on Aug 27, 2025
  4. changed the title [-]Don't change the order of user-entered search syntax[/-] [+][$250] Don't change the order of user-entered search syntax[/+] on Aug 27, 2025
  5. melvin-bot commented on Aug 27, 2025

    @melvin-bot
  6. added
    Help WantedApply this label when an issue is open to proposals by contributors
    on Aug 27, 2025
  7. melvin-bot commented on Aug 27, 2025

    @melvin-bot

    Triggered auto assignment to Contributor-plus team member for initial proposal review - @shubham1206agra (External)

  8. mohammadjafarinejad commented on Aug 28, 2025

    @mohammadjafarinejad
    Contributor

    🚨 Edited by proposal-police: This proposal was edited at 2025-08-28 09:45:31 UTC.

    Proposal

    Please re-state the problem that we are trying to solve in this issue.

    Don't change the order of user-entered search syntax

    What is the root cause of that problem?

    When users submit a search, the query gets processed through multiple functions that rebuild it in a fixed order rather than preserving the original input sequence.

    1. In SearchPageHeaderInput.tsx, getQueryWithUpdatedValues() calls buildSearchQueryString(), which processes parts in a fixed order.
      for (const [, key] of Object.entries(CONST.SEARCH.SYNTAX_ROOT_KEYS)) {
    2. When rendering results, buildUserReadableQueryString() also enforces a fixed order for the title.
      let title = status ? `type:${getUserFriendlyValue(type)} status:${Array.isArray(status) ? status.join(',') : status}` : `type:${getUserFriendlyValue(type)}`;
      if (groupBy) {
      title += ` group-by:${getUserFriendlyValue(groupBy)}`;
      }

    As a result, queries are reconstructed in a constant order instead of matching the user’s input.

    What changes do you think we should make in order to solve the problem?

    Add a helper function in SearchQueryUtils.ts to sort query parts based on original input order (the exact sorting logic can be refined in the PR if needed)

    function sortQueryPartsToMatchInputOrder(parts: string[], originalQuery: string) {
        const sortedFriendlyKeys = originalQuery.split(' ').map((part) => {
            const key = part.split(':').at(0);
            return getUserFriendlyKey(key as SearchFilterKey);
        });
    
        const sortedParts = parts.sort((a, b) => {
            const aKey = a.split(':').at(0) ?? '';
            const bKey = b.split(':').at(0) ?? '';
            const aFriendly = getUserFriendlyKey(aKey as SearchFilterKey);
            const bFriendly = getUserFriendlyKey(bKey as SearchFilterKey);
            const aIndex = sortedFriendlyKeys.indexOf(aFriendly);
            const bIndex = sortedFriendlyKeys.indexOf(bFriendly);
    
            // If both keys are found in the original query, sort by their position
            if (aIndex !== -1 && bIndex !== -1) {
                return aIndex - bIndex;
            }
            // If only one is found, prioritize it
            if (aIndex !== -1) {
                return -1;
            }
            if (bIndex !== -1) {
                return 1;
            }
            // If neither is found, maintain the current order
            return 0;
        });
    
        return sortedParts;
    }

    If we want routing to stay consistent with what the user sees in the search input, then both buildSearchQueryString() and buildUserReadableQueryString() should apply sorting.

    Apply sorting

    1. In buildSearchQueryString:
    // Sort queryParts to match the original input order
    if (queryJSON?.inputQuery) {
        queryParts = sortQueryPartsToMatchInputOrder(queryParts, queryJSON.inputQuery);
    }
    1. In buildUserReadableQueryString(): Replace the current let title += approach with an array-based approach:
    const titleParts: string[] = [];
    // ... collect parts into titleParts array instead of concatenating to title
    
    // Sort titleParts to match the original input order  
    if (queryJSON.inputQuery) {
        titleParts = sortQueryPartsToMatchInputOrder(titleParts, queryJSON.inputQuery);
    }
    
    return titleParts.join(' ');

    If only the display order matters (and routing can stay as-is), then we don’t need to update buildSearchQueryString().
    Instead, we can save the raw queryString at submitSearch() and pass it to buildUserReadableQueryString() to control sorting:

    if (queryJSON.inputQuery) {
        titleParts = sortQueryPartsToMatchInputOrder(titleParts, originalInputQuery);
    }

    The downside of this approach is that the routing won’t preserve the user’s preferred order.
    This means if the user refreshes the page, the query order will reset to the default.

    What specific scenarios should we cover in automated tests to prevent reintroducing this issue in the future?

    Add tests to confirm that getQueryWithUpdatedValues() and buildUserReadableQueryString() always keep the same order as the input query.

    What alternative solutions did you explore? (Optional)

    N/A

  9. sofi-a commented on Aug 28, 2025

    @sofi-a
    Contributor

    🚨 Edited by proposal-police: This proposal was edited at 2025-08-28 03:01:20 UTC.

    Proposal

    Please re-state the problem that we are trying to solve in this issue.

    Avoid re-ordering user-entered search syntax

    What is the root cause of that problem?

    The buildSearchQueryString and buildUserReadableQueryString functions use a hardcoded order when computing the query string and search text, respectively.

    for (const [, key] of Object.entries(CONST.SEARCH.SYNTAX_ROOT_KEYS)) {
    const existingFieldValue = queryJSON?.[key];
    const queryFieldValue = existingFieldValue ?? defaultQueryJSON?.[key];
    if (queryFieldValue) {
    if (Array.isArray(queryFieldValue)) {
    queryParts.push(`${key}:${queryFieldValue.join(',')}`);
    } else {
    queryParts.push(`${key}:${queryFieldValue}`);
    }
    }
    }

    let title = status
    ? `type:${getUserFriendlyValue(type)} status:${Array.isArray(status) ? status.map(getUserFriendlyValue).join(',') : getUserFriendlyValue(status)}`
    : `type:${getUserFriendlyValue(type)}`;
    if (groupBy) {
    title += ` group-by:${getUserFriendlyValue(groupBy)}`;
    }
    if (policyID && policyID.length > 0) {
    title += ` workspace:${policyID.map((id) => sanitizeSearchValue(policies?.[`${ONYXKEYS.COLLECTION.POLICY}${id}`]?.name ?? id)).join(',')}`;
    }

    What changes do you think we should make in order to solve the problem?

    1. Update the searchParser to include tracking metadata for the positions of search filters.
    // per-parser initializer (code executed before every parse)
    {
      ...
      const positions = {};
    
      function applyDefaults(filters) {
        return {
          ...defaultValues,
          filters,
          positions
        };
      }
      ...
    }
    1. Use the location() special function to get the start and end positions of the current rule match for both the default and standard filter keys.
    filterKey
      = k:key {
          nameOperator = (k === "from" || k === "to");
          const pos = location();
          positions[k] = pos;
          return k;
    }
    
    defaultKey "default key"
      = k:(type / status / sortBy / sortOrder / policyID / groupBy) {
          const pos = location();
          positions[k] = pos;
          return k;
      }
    1. Update the type declaration for SearchQueryJSON.
    type Position = {
        offset: number;
        line: number;
        column: number;
    };
    
    type Range = {
        start: Position;
        end: Position;
    };
    
    type Positions = Record<SearchFilterKey, Range>;
    
    type SearchQueryJSON = {
        ...
        positions: Positions;
    } & SearchQueryAST;
    1. Create a function getFiltersOrder that returns an array of filter keys ordered according to their positions in the query.
    function getFiltersOrder(positions: Positions): string[] {
        return Object.entries(positions)
            .sort(([, a], [, b]) => a.start.offset - b.start.offset)
            .map(([key]) => key);
    }
    1. Create a function sortQueryPartsByFilterOrder that sorts query parts based on the position of filters.
    function sortQueryPartsByFilterOrder(queryParts: string[], filtersOrder: string[]) {
        return [...queryParts].sort((a, b) => {
            const aKey = a.split(':').at(0) ?? '';
            const bKey = b.split(':').at(0) ?? '';
            const aIdx = filtersOrder.indexOf(aKey);
            const bIdx = filtersOrder.indexOf(bKey);
    
            // If both keys are found, sort by their order in inputQuery
            if (aIdx !== -1 && bIdx !== -1) {
                return aIdx - bIdx;
            }
            // If only one is found, that one comes first
            if (aIdx !== -1) {
                return -1;
            }
            if (bIdx !== -1) {
                return 1;
            }
            // Otherwise, keep original order
            return 0;
        });
    }
    1. Update the buildSearchQueryString function to return a query string sorted by the positions of the search filters.
    function buildSearchQueryString(queryJSON?: SearchQueryJSON) {
    ...
        const filtersOrder = getFiltersOrder(queryJSON.positions);
        return sortQueryPartsByFilterOrder(queryParts, queryJSON.filtersOrder).join(' ');
    }
    1. Refactor the buildUserReadableQueryString function to order the search text properly.
    function buildUserReadableQueryString(queryJSON: SearchQueryJSON, ...restArgs) {
        ...
        const queryParts = [`type:${getUserFriendlyValue(type)}`];
    
        if (status) {
            queryParts.push(`status:${Array.isArray(status) ? status.map(getUserFriendlyValue).join(',') : getUserFriendlyValue(status)}`);
        }
    
        if (groupBy) {
            queryParts.push(`group-by:${getUserFriendlyValue(groupBy)}`);
        }
    
        if (policyID && policyID.length > 0) {
            queryParts.push(`workspace:${policyID.map((id) => sanitizeSearchValue(policies?.[`${ONYXKEYS.COLLECTION.POLICY}${id}`]?.name ?? id)).join(',')}`);
        }
    
        for (const filterObject of filters) {
            ...
            queryParts.push(buildFilterValuesString(getUserFriendlyKey(key), displayQueryFilters).trim());
        }
    
        const filtersOrder = getFiltersOrder(queryJSON.positions);
        return sortQueryPartsByFilterOrder(queryParts, filtersOrder).join(' ');
    }

    What specific scenarios should we cover in automated tests to prevent reintroducing this issue in the future?

    N/A

    What alternative solutions did you explore? (Optional)

    N/A

  10. 71 remaining items

  11. ShridharGoel commented on Dec 10, 2025

    @ShridharGoel
    Contributor

    @JmillsExpensify This seems to be due for payment.

  12. JmillsExpensify commented on Dec 11, 2025

    @JmillsExpensify
    ContributorAuthor

    Ah interesting. Looks like the automation failed.

  13. JmillsExpensify commented on Dec 11, 2025

    @JmillsExpensify
    ContributorAuthor

    Payment summary:

  14. JmillsExpensify commented on Dec 11, 2025

    @JmillsExpensify
    ContributorAuthor

    Reviewers - let me know if you disagree, though one of your reviewed the proposals and another reviewed the PR, so that seemed like the fair thing to do.

  15. JmillsExpensify commented on Dec 11, 2025

    @JmillsExpensify
    ContributorAuthor

    Regression test added and all contributors paid via ND, so closing.

  16. eh2077 commented on Dec 11, 2025

    @eh2077
    Contributor

    @JmillsExpensify Thanks for checking this. But I think I should be paid same as contributor. We almost ended up with a new solution of the original proposal, see #69394 (comment) as well. Just wanted to highlight the work we have done and I'm not going to deny the effort previous reviewer put in for the task.

  17. JmillsExpensify commented on Dec 11, 2025

    @JmillsExpensify
    ContributorAuthor

    That sounds reasonable, I've updated my comment.

  18. eh2077 commented on Dec 11, 2025

    @eh2077
    Contributor

    Thank you so much 🙇

  19. twisterdotcom commented on Dec 11, 2025

    @twisterdotcom
    Contributor

    @ShridharGoel has submitted payment for this report on ND here.

    SO: https://stackoverflowteams.com/c/expensify/questions/7582

    Zapier Logs Run ID: 00040eee-8e29-a187-556b-c15c49e2f3e4
  20. garrettmknight commented on Dec 18, 2025

    @garrettmknight
    Contributor

    $500 approved for @eh2077

  21. garrettmknight commented on Apr 2, 2026

    @garrettmknight
    Contributor

    $250 approved for @shubham1206agra

  22. thesahindia commented on May 11, 2026

    @thesahindia
    Member

    The fix for this caused #87285. Explanation here.

  23. sobitneupane commented on May 25, 2026

    @sobitneupane
    Contributor

    The PR for this issue caused #79952. When searching in #chat, the user was unintentionally being directed to type:expense in:#chat instead of type:chat in:#chat.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

BugSomething is broken. Auto assigns a BugZero manager.ExternalAdded to denote the issue can be worked on by a contributorMonthlyKSv2ReviewingHas a PR in review

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions