Repository navigation
Open
Labels
Description
Description
On windows server 2022, when using docker login so I can push an image to ecr, I get this weird error:
Error saving credentials: error storing credentials - err: exit status 1, out: `The stub received bad data.`
There's lots of recommendations to clear or remove the .docker\config.json file, but that doesn't fix the issue.
Reproduce
- Start a windows 2022 server ec2 instance in aws.
- Install docker for windows
- Switch to windows containers
- (Get-ECRLoginCommand).Password | docker login --username AWS --password-stdin my-ecr-am.amazonaws.example.com
Expected behavior
No response
docker version
Client:
Cloud integration: v1.0.33
Version: 24.0.2
API version: 1.43
Go version: go1.20.4
Git commit: cb74dfc
Built: Thu May 25 21:53:15 2023
OS/Arch: windows/amd64
Context: default
Server: Docker Desktop 4.20.0 (109717)
Engine:
Version: 24.0.2
API version: 1.43 (minimum version 1.24)
Go version: go1.20.4
Git commit: 659604f9
Built: Thu May 25 21:52:13 2023
OS/Arch: windows/amd64
Experimental: falsedocker info
Client:
Version: 24.0.2
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.10.5
Path: C:\Program Files\Docker\cli-plugins\docker-buildx.exe
compose: Docker Compose (Docker Inc.)
Version: v2.18.1
Path: C:\Program Files\Docker\cli-plugins\docker-compose.exe
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.0
Path: C:\Program Files\Docker\cli-plugins\docker-dev.exe
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.19
Path: C:\Program Files\Docker\cli-plugins\docker-extension.exe
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v0.1.0-beta.4
Path: C:\Program Files\Docker\cli-plugins\docker-init.exe
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: C:\Program Files\Docker\cli-plugins\docker-sbom.exe
scan: Docker Scan (Docker Inc.)
Version: v0.26.0
Path: C:\Program Files\Docker\cli-plugins\docker-scan.exe
scout: Command line tool for Docker Scout (Docker Inc.)
Version: v0.12.0
Path: C:\Program Files\Docker\cli-plugins\docker-scout.exe
Server:
Containers: 8
Running: 0
Paused: 0
Stopped: 8
Images: 74
Server Version: 24.0.2
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: ics internal l2bridge l2tunnel nat null overlay private transparent
Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: process
Kernel Version: 10.0 20348 (20348.1.amd64fre.fe_release.210507-1500)
Operating System: Microsoft Windows Server Version 21H2 (OS Build 20348.1547)
OSType: windows
Architecture: x86_64
CPUs: 4
Total Memory: 15.86GiB
Name: EC2AMAZ-H68PGQ1
ID: 3cc58b8c-583d-4ff9-9632-529b7596c882
Docker Root Dir: C:\ProgramData\Docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: falseAdditional Info
No response
Reacted by Andrea Scarpino
Activity
djeikyb commentedon Jun 1, 2023
I finally found someone recommending to run the system file checker. It worked!
Sorry for the spam, I just spent far too long on this. Hopefully this helps someone out?
thaJeztah commentedon Jun 1, 2023
Interesting; hadn't heard of that one (but admitted, I'm not on Windows myself). IIUC that command is to fix issues with your Windows installation (as a whole?)
djeikyb commentedon Jun 1, 2023
Myself, I'm on a mac 😄 Just over here tryna deploy legacy asp.net applications lol.
And yeah, I think there must be something weird with the win server 2022 image from aws. The problem was only manifesting when trying to use
docker login, but thatsfcfixed things up, i'm pretty sure means it wasn't related to docker.thaJeztah commentedon Jun 1, 2023
Thanks! I'll try to keep this one in the back of my mind in case I see someone reporting something like this!
crazy-max commentedon Jul 6, 2023
Hum also encounter this one on windows runners on GHA: https://github.com/docker/login-action/actions/runs/5472897107/jobs/9965664777#step:4:15
thaJeztah commentedon Jul 6, 2023
Let me post that output here as well (easier to search);
thaJeztah commentedon Jul 6, 2023
Some mention in Go code here; https://github.com/minepkg/minepkg/blob/c2146e9b476e6e2ed46a23f21b89a9aaff132c25/internals/auth/microsoft.go#L23-L32
Error-code (not the canonical source of truth); https://github.com/mickael-kerjean/filestash/blob/c2059c839d516c2ee56570d1d636c97a9c807278/vendor/github.com/hirochachacha/go-smb2/internal/erref/ntstatus.go#L2938
But looks to be documented here; https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55
yerlibilgin commentedon Oct 27, 2024
Hi people. @djeikyb I am sorry to see that you have closed the report. After all,
sfc /scannowis not a general solution, but a workaround. In recent windows versions, this has been a quite problem where people devise custom workarounds for it (deploying alternative credential helpers, which, I believe, isn't a quite sustainable solution). So I would love to see it reopened and solved.I believe the issue is that the docker windows credential manager doesn't like looong stuff. Maybe they forgot to include tokens, and focused only in human passwords of some maximum length.
djeikyb commentedon Oct 30, 2024
@yerlibilgin tbh i opened this issue after i already had a solution just to document (a) the problem (b) the solution i painstakingly discovered. if you think this is a problem that should be solved in the docker codebase, please do open an issue! and tag this one in for searchability!
criscola commentedon Nov 21, 2024
Unfortunately
sfc /scannowdid not fix the issue for me.thaJeztah commentedon Nov 21, 2024
I'm re-reading the issue, and the comment I left further up #346, and I think I have a slightly clearer picture of the issue (bear with me; I'm not a Windows user), although I don't exactly understand how the
sfc /scannowrelates.The original issue was reported with this command;
This command;
(Get-ECRLoginCommand).Password, which produces a token / passworddocker login, passing the token / password to it throughstdinmy-ecr-am.amazonaws.example.com) to verify the credentials~/.docker/config.json)wincredcredentials-helper (https://github.com/docker/docker-credential-helpers/tree/master/wincred) which stores it in the Windows Credentials Store.From my understanding, the last part is where things fail, and from #346 it looks that (as also mentioned by @yerlibilgin #346) the
The stub received bad data.error is produced by the Windows API to store those credentials if the data is too large.I'm not sure if there's good ways around this; the code in the
minepkg/minepkgproject (from trying to read their changes) worked around this issue by stripping the token from data that can be omitted, but in their case it's a token they produce and therefore were able to mutate.In the docker CLI + credentials helper case, that's not a real option; the CLI treats the password (or token passed as password) as opaque data; it verified that data is correct against the registry in which case it stores it as-is. And, in this case, the token is produced by
(Get-ECRLoginCommand).Password, which is a 3rd party tool, so we cannot mutate that data.So the only solution I could see here is to either
(Get-ECRLoginCommand).Passwordto produce a token that's smaller (I guess it may depend on the situation what information is captured in the token)That last change would have to be implemented in either the credentials helper (https://github.com/docker/docker-credential-helpers/tree/master/wincred), or the Go library that's used to interact with the Windows Credentials store (github.com/danieljoos/wincred).
I will reopen this ticket, but move it to the https://github.com/docker/docker-credential-helpers repository's issue tracker, because the underlying issue here does not appear to be in the docker CLI itself.
Let me also cc @austinvazquez in case he has more insight in the ECRLoginCommand to see if there's options for it to produce a token that's smaller (omitting data that may not be essential?)
fartosh commentedon Jan 10, 2025
@thaJeztah thanks for sharing, it helped me find the solution - in my case I simply removed
"credsstore": "wincred"from%APPDATA%\helm\registry\config.jsonbecause I didn't need it.I can confirm that. Today I faced the same issue with
helm registry login public.ecr.aws. I checked that in my%APPDATA%\helm\registry\config.jsonthere was Windows Credential Manager configured as a store.Not really an official documentation but if you go to Credential Manager and try to add a generic credential, the password field will remain blank if you try to paste the generated token but it will accept anything that is significantly shorter. On my machine (Windows 10 22H2) the limit is set to 259 characters, the password field will not accept any string equal to or longer than 260 characters.
davecranwell commentedon Jan 14, 2025
I got this for the first time today. I was running WSL2 and trying to build for ECR from there, when I experienced the problem.
All the solutions I found suggested removing the
credsStorevalue of your dockerconfig.json. But it wasn't the Windows version of that file (at%UserProfile%\.docker\config.json) I had to change but rather the Linux one, on WSL2 at `~/.docker/config.json.In my case I found that
credsStorein the linux file had the valuedesktop.exeand removing this line entirely solved my problems! Changing the windows version just resulted in the same data being added back to it every time docker desktop restarted.paulchambers commentedon Feb 20, 2025
Removing the credsStore entry didn't work here. I had to delete "docker\Docker\resources\bin\docker-credential-wincred.exe" and of course that comes back on each update.
Is there any progress in a permanent fix for this?
keirans commentedon Jul 31, 2025
Thanks so much for this.
This also worked for me, I just renamed it and ran my commands to auth again and it worked a treat.
ShachafGoldstein commentedon Feb 12, 2026
Something less permeant if anyone prefers - removed the bin folder for the running terminal for the login or relevant commands:
TBBle commentedon Feb 19, 2026
Sadly, all credential types in the Windows Credential Store API share the same 2.5kb size limit per
CredentialBlobSizedocs:So the safest and most-extensible approach I see here would be to store the credential in a file or the registry or something, encrypted, and store the key used for the encryption in the Credentials Store.
As a less-safe approach, credentials can also have up to 64 attributes, and each of those can be up to 256 bytes, so that's another 16kb of space to work with. I don't know off-hand how big the ECS tokens are though. (Edit: Had one lying around, it was under 4kb. So this is potentially feasible, or at least kicks the problem further down the road.)