Skip to content

Fix sanitize_filename exceeding max_len when escaping reserved names (#83) - #84

Open
shiylerde wants to merge 1 commit into
thombashi:masterfrom
shiylerde:fix/83-reserved-name-max-len
Open

shiylerde wants to merge 1 commit into
thombashi:masterfrom
shiylerde:fix/83-reserved-name-max-len

Conversation

@shiylerde

Copy link
Copy Markdown

Summary

Fixes #83.

sanitize_filename() can return a name that is longer than the requested max_len and therefore fails is_valid_filename(..., max_len=...) when the input is a Windows reserved name.

Reproduction (before this change)

>>> from pathvalidate import sanitize_filename, is_valid_filename
>>> sanitize_filename("CON", platform="windows", max_len=3)
'CON_'                          # 4 bytes although max_len=3
>>> is_valid_filename(_, platform="windows", max_len=3)
False

The same happens for PRN / AUX / NUL with max_len=3 (e.g. 'PRN_'), for COM1 / LPT1 with max_len=4 (e.g. 'COM1_'), etc. With an extension it can also leave a dangling fragment, e.g. sanitize_filename("CON.txt", platform="windows", max_len=5) returned 'CON_.t' (6 bytes, invalid).

Root cause

FileNameSanitizer.sanitize() processes the name in this order:

  1. replace invalid characters;
  2. truncate to max_len (pathvalidate/_filename.py:80);
  3. validate, and when the name is a reserved name the handler appends an escape marker — the default ReservedNameHandler.add_trailing_underscore appends "_" (pathvalidate/_filename.py:85-90).

Step 3 grows the name after it was truncated, and there is no subsequent length check, so the escaped name can exceed max_len. Simply truncating the escaped name again does not fully solve it: "CON_"[:3] == "CON" is still the reserved name, and cutting in the middle of an extension can leave a trailing "." / space, which is invalid on Windows.

Fix

After applying the reserved-name replacement, re-fit the result to max_len with a new _fit_reserved_name() helper:

  • truncate to max_len bytes using the existing byte-aware truncate_str;
  • while the result is not a valid filename, shorten it — on Windows first strip trailing spaces/periods (which drops a dangling extension cut), otherwise remove one more byte;
  • each iteration removes at least one byte, so the loop always terminates;
  • if no non-empty name can fit within the limit (for example when every single-byte name is a reserved name), raise ValidationError(INVALID_LENGTH) instead of silently returning an over-limit name.

The "is this still reserved / valid?" decision is reused from the existing validator via self._validator.is_valid(...), so the reserved-name rules stay in a single place.

FilePathSanitizer benefits from the same fix automatically, because it sanitizes each path component through a FileNameSanitizer.

Concrete changes

  • pathvalidate/_filename.py
    • call self._fit_reserved_name(...) after the reserved-name re.sub(...) in sanitize();
    • add the FileNameSanitizer._fit_reserved_name() helper.
  • test/test_filename.py — regression tests covering:
    • the issue's cases (CON / PRN / AUX / NUL with max_len=3, COM1 / LPT1 with max_len=4);
    • reserved names with an extension and a very small max_len;
    • a sweep over every WIN_RESERVED_FILE_NAMES on windows/universal across several max_len values and handlers (default, leading-underscore, trailing-underscore), asserting the result is always within the byte limit and valid;
    • validate_after_sanitize=True (this used to raise INVALID_AFTER_SANITIZE);
    • the impossible-limit case raising ValidationError(INVALID_LENGTH).

Potential impact / behavior

  • When max_len already accommodates the escape marker (the normal case, including the default 255), the output is unchanged: "CON" -> "CON_", "CON.txt" -> "CON_.txt".
  • Only the previously-broken case — where the marker pushed the name past max_len — changes, from returning an invalid over-limit name to returning a valid name within the limit (e.g. "CON" with max_len=3 -> "CO").
  • The as_is reserved-name handler (which intentionally keeps reserved names) is untouched.
  • No public API signature changes. A ValidationError(INVALID_LENGTH) is raised only in the pathological situation where no valid non-empty name can fit in the requested limit.

Verification

  • pytest test/ -> 4788 passed, 14 skipped (the skips are pre-existing, platform-specific).
  • mypy pathvalidate -> Success: no issues found in 14 source files.
  • ruff format --check -> clean; ruff check reports no findings in the changed lines (identical pre-existing warning count before/after).
  • Manual reproduction of every example above now returns a name with len(name.encode(fs_encoding)) <= max_len for which is_valid_filename(...) is True.

)

FileNameSanitizer.sanitize() truncated the filename to max_len first and
then let the reserved-name handler append an escape marker (e.g. "CON"
-> "CON_"), which grew the name back beyond max_len with no subsequent
length check. The returned name therefore failed
is_valid_filename(..., max_len=...):

    sanitize_filename("CON", platform="windows", max_len=3)
    # before: "CON_"  (4 bytes, invalid for max_len=3)

Re-fit the escaped name to max_len and shorten it until it is a valid,
non-reserved filename, trimming trailing spaces/periods on Windows when
an extension was cut off by truncation. If no non-empty name can fit
within the limit (e.g. every single-byte name is reserved), raise
ValidationError(INVALID_LENGTH) instead of returning an over-limit name.

FilePathSanitizer benefits from the same fix because it sanitizes each
path component through FileNameSanitizer.

Fixes thombashi#83

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: sanitize_filename exceeds max_len after replacing a reserved Windows name

1 participant