Skip to content

gh-119670: Add force keyword only argument to shlex.quote#148846

Open
jb2170 wants to merge 17 commits into
python:mainfrom
jb2170:shlex-quote-force
Open

gh-119670: Add force keyword only argument to shlex.quote#148846
jb2170 wants to merge 17 commits into
python:mainfrom
jb2170:shlex-quote-force

Conversation

@jb2170
Copy link
Copy Markdown
Contributor

@jb2170 jb2170 commented Apr 21, 2026

Closes #119670
Supersedes my out-of-date PR #119674

We use force instead of always as a word more consistent with other programming jargon (like in rm -f the f is for 'force') as discussed

Commits

  • ddbe3a5: Add base implementation.
  • 0e722d6: Make shlex.quote's kwargs kwargs-only. Backwards compatible. There are propositions to add a single-quote-double-quote switch, which I'm ambivalent about, but to future-proof against hiccups of people passing force as a positional and it ending up being used as the wrong kwarg, we make the kwargs kwargs-only, eg like json.loads.
  • fd4af18: Add tests. I've tried to be pretty thorough. Their purposes are commented on.
  • 78b6f3b: Update docs. Contains explanation and examples.
  • 762999d: Add blurb entry.
  • 2a301a5: Add whatsnew entry.

As mentioned I'm also going to open an issue a discussion thread about shlex.quote returning "''" for falsey non-str data. This PR and that discussion seem independent of each other, that is this PR doesn't need to be held back until that discussion is addressed.


📚 Documentation preview 📚: https://cpython-previews--148846.org.readthedocs.build/

@jb2170
Copy link
Copy Markdown
Contributor Author

jb2170 commented Apr 21, 2026

@vstinner pong 😅

Copy link
Copy Markdown
Member

@johnslavik johnslavik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment thread Lib/shlex.py Outdated
Comment thread Lib/test/test_shlex.py Outdated
Comment thread Lib/test/test_shlex.py Outdated
@johnslavik
Copy link
Copy Markdown
Member

johnslavik commented Apr 22, 2026

Hmm I also have a hunch that this might be worth calling out in What's New. Nvm it's already there, great job 🚀

@jb2170
Copy link
Copy Markdown
Contributor Author

jb2170 commented Apr 24, 2026

Looks ready (for review again?) to me 😎

Copy link
Copy Markdown
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@picnixz: Do you want to double check? You reviewed the previous PR gh-119674.

Copy link
Copy Markdown
Member

@picnixz picnixz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor tweaks but please add more tets with quotes inside.

Comment thread Doc/library/shlex.rst Outdated
Comment thread Doc/library/shlex.rst Outdated
Comment thread Doc/library/shlex.rst
Comment thread Doc/whatsnew/3.15.rst Outdated
Comment thread Lib/shlex.py Outdated
Comment thread Lib/shlex.py Outdated
Comment thread Lib/shlex.py Outdated
Comment thread Lib/test/test_shlex.py
Comment thread Misc/NEWS.d/next/Library/2026-04-21-06-30-59.gh-issue-119670.pMWZfY.rst Outdated
@jb2170 jb2170 force-pushed the shlex-quote-force branch from a9e06cd to e7bf496 Compare April 28, 2026 07:44
@picnixz
Copy link
Copy Markdown
Member

picnixz commented Apr 28, 2026

Please avoid force-pushing. It's extremely hard to make reviews otherwise.

@jb2170 jb2170 force-pushed the shlex-quote-force branch from e7bf496 to e3d9b05 Compare April 28, 2026 08:38
@jb2170
Copy link
Copy Markdown
Contributor Author

jb2170 commented Apr 28, 2026

Sorry for the force pushes but the 'commit suggestions' commits on GitHub came through as CRLF formatted instead of LF. I don't know if that's a GitHub Microsoft issue, or your IDE settings, or I need to git config autocrlf but hey I got there in the end.

$ git log d000ccd...e3d9b05 is what has changed, addressing the review. The only change I disagreed with is the wording of the .. versionadded. The existing one seems to be worded more similarly to the others.

@picnixz
Copy link
Copy Markdown
Member

picnixz commented Apr 28, 2026

I think it's a GH issue because other reviews had the same issue...

@jb2170
Copy link
Copy Markdown
Contributor Author

jb2170 commented Apr 28, 2026

Ready

@picnixz picnixz self-requested a review April 28, 2026 15:30
@jb2170
Copy link
Copy Markdown
Contributor Author

jb2170 commented May 28, 2026

@picnixz 30 day courtesy ping 👀

Comment thread Lib/shlex.py Outdated
@jb2170 jb2170 requested a review from picnixz June 3, 2026 01:00
Copy link
Copy Markdown
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jb2170: Please move the "What's New in Python" documentation as I explained in my comment below.

Comment thread Doc/whatsnew/3.15.rst Outdated
Hugo van Kemenade in :gh:`148100`.)


* :mod:`shlex`:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hum. This change will land in Python 3.16 and so Doc/whatsnew/3.16.rst should be modified instead. Moreover, your change adds an entry in Deprecations which is wrong. You should add an entry at https://docs.python.org/dev/whatsnew/3.15.html#improved-modules : add "shlex" sub-section in the Improved modules section.

Copy link
Copy Markdown
Contributor Author

@jb2170 jb2170 Jun 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I rebased to get the 3.16 whatsnew file. Should I have merged instead of rebase eep
(My thinking was that merging main into this branch would change all the commit hashes for main since this branch, when nope it was the right one ffs 😭 )

Copy link
Copy Markdown
Contributor Author

@jb2170 jb2170 Jun 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$ git range-diff d206d42..7fa8a7a 6453065..a030aa4 shows everything is the same before and after the rebase

jb2170 added 3 commits June 3, 2026 13:59
There are propositions to add a single-quote-double-quote switch
(pythongh-90630), so to avoid hiccups of people passing `force` as a
positional and it being used for the single-double switch, we make
kwargs kwargs-only.
Test special cases of strings that don't need quoting, do need quoting,
do use `force`, don't use `force` etc. I've tried to be exhaustive.
jb2170 and others added 13 commits June 3, 2026 13:59
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
The comments for why these tests exist can always be found in the git history

Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
Intended for another branch shlex-quote-typeerror

Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
…view

Show off the default behavior as `force=False`
Try not to use don't and we're and should've etc apostrophes.
Add test to ensure string with single quote inside it (without which the
string wouldn't need quoting) is quoted even though `force=False`.
@jb2170 jb2170 force-pushed the shlex-quote-force branch from 7fa8a7a to a030aa4 Compare June 3, 2026 12:59
@read-the-docs-community
Copy link
Copy Markdown

Documentation build overview

📚 cpython-previews | 🛠️ Build #32972607 | 📁 Comparing 7781ca0 against main (0b9146e)

  🔍 Preview build  

161 files changed · + 5 added · ± 156 modified

+ Added

± Modified

@jb2170
Copy link
Copy Markdown
Contributor Author

jb2170 commented Jun 3, 2026

Really sorry for the rebase ffs, worst PR ever! 😭
I normally rebase instead of merge. I thought merging main would cause this exact issue but nope the other way round
$ git range-diff d206d42..7fa8a7a 6453065..a030aa4 shows everything is the same before and after the rebase, there's nothing sneaky going on

@jb2170 jb2170 requested a review from johnslavik June 3, 2026 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

shlex.quote: Add force keyword argument

4 participants