Repository navigation
Description
Describe the bug
A skippable frame whose Frame_Size is between 0xFFFFFFF8 and 0xFFFFFFFF is refused with frameParameter_unsupported by ZSTD_findFrameCompressedSize and ZSTD_decompress. ZSTD_decompressStream skips the same frame without error. RFC 8878, section 3.1.2, allows User_Data of up to 2^32-1 bytes.
Cause: readSkippableFrameSize checks that sizeU32 + 8 fits in 32 bits (L594-L595), although it then computes the frame size in size_t. ZSTD_findDecompressedSize, ZSTD_decompressBound and ZSTD_readSkippableFrame go through the same function and refuse the frame too (output below).
To Reproduce
The PoC builds one skippable frame with Frame_Size = 0xFFFFFFF8 (a 4 GiB buffer from calloc).
make -C lib libzstd.acc -O2 -Ilib poc.c lib/libzstd.a -o poc && ./poc- Output:
frame of 4294967296 bytes (User_Data 4294967288 bytes)
ZSTD_findFrameCompressedSize: Unsupported frame parameter
ZSTD_decompress: Unsupported frame parameter
ZSTD_decompressStream: 0, read 4294967296 of 4294967296
With -DSIZE=0xFFFFFFF0u, just below the limit, every call accepts the frame.
The same frame passed to the other functions that read a skippable frame's size (ZSTD_STATIC_LINKING_ONLY), on dev and v1.5.7:
ZSTD_findDecompressedSize: ZSTD_CONTENTSIZE_ERROR
ZSTD_decompressBound: ZSTD_CONTENTSIZE_ERROR
ZSTD_readSkippableFrame: Src size is incorrect
poc.c
#include <stdio.h>
#include <stdlib.h>
#include "zstd.h"
#ifndef SIZE
#define SIZE 0xFFFFFFF8u
#endif
int main(void)
{
size_t len = 8 + (size_t) SIZE;
unsigned char *f = calloc(len, 1), out[1];
if (f == NULL) { printf("no memory for %zu bytes\n", len); return 1; }
f[0] = 0x50; f[1] = 0x2A; f[2] = 0x4D; f[3] = 0x18; /* Magic_Number 0x184D2A50 */
f[4] = (unsigned char) SIZE; f[5] = (unsigned char) (SIZE >> 8); /* Frame_Size, little-endian */
f[6] = (unsigned char) (SIZE >> 16); f[7] = (unsigned char) (SIZE >> 24);
size_t fs = ZSTD_findFrameCompressedSize(f, len);
size_t d = ZSTD_decompress(out, sizeof out, f, len);
printf("frame of %zu bytes (User_Data %u bytes)\n", len, SIZE);
printf("ZSTD_findFrameCompressedSize: %s\n", ZSTD_isError(fs) ? ZSTD_getErrorName(fs) : "accepted");
printf("ZSTD_decompress: %s\n", ZSTD_isError(d) ? ZSTD_getErrorName(d) : "accepted");
ZSTD_DStream *zds = ZSTD_createDStream();
ZSTD_inBuffer in = { f, len, 0 };
ZSTD_outBuffer o = { out, sizeof out, 0 };
size_t r = ZSTD_decompressStream(zds, &o, &in);
printf("ZSTD_decompressStream: %s, read %zu of %zu\n", ZSTD_isError(r) ? ZSTD_getErrorName(r) : "0", in.pos, len);
ZSTD_freeDStream(zds);
free(f);
return 0;
}Expected behavior
One-shot decoding and the frame-size functions accept the frame, as streaming does: ZSTD_findFrameCompressedSize returns 4294967296, ZSTD_decompress, ZSTD_findDecompressedSize and ZSTD_decompressBound return 0, and ZSTD_readSkippableFrame returns the 4294967288 bytes of User_Data.
Desktop:
- OS: Ubuntu 24.04.4 LTS
- Version: dev 49cf517, v1.5.7 and v1.4.9 (the check is the same in all three)
- Compiler: gcc 13.3.0
- Flags: default
lib/Makefileflags, PoC built with-O2 - Other relevant hardware specs: x86_64
- Build system: Makefile
Additional context
Suggested fix: compare in size_t. srcSize >= 8 is checked just before, so neither side wraps, also with a 32-bit size_t. A truncated frame then gives srcSize_wrong, as it already does below the limit.
--- a/lib/decompress/zstd_decompress.c
+++ b/lib/decompress/zstd_decompress.c
@@ -591,12 +591,8 @@ static size_t readSkippableFrameSize(void const* src, size_t srcSize)
RETURN_ERROR_IF(srcSize < ZSTD_SKIPPABLEHEADERSIZE, srcSize_wrong, "");
sizeU32 = MEM_readLE32((BYTE const*)src + ZSTD_FRAMEIDSIZE);
- RETURN_ERROR_IF((U32)(sizeU32 + ZSTD_SKIPPABLEHEADERSIZE) < sizeU32,
- frameParameter_unsupported, "");
- { size_t const skippableSize = skippableHeaderSize + sizeU32;
- RETURN_ERROR_IF(skippableSize > srcSize, srcSize_wrong, "");
- return skippableSize;
- }
+ RETURN_ERROR_IF(sizeU32 > srcSize - skippableHeaderSize, srcSize_wrong, "");
+ return skippableHeaderSize + sizeU32;
}
/*! ZSTD_readSkippableFrame() :Found by genproof.com Formal Verification Agent. Reviewed and approved by a human.
Activity