Skip to content

One-shot decoding refuses some skippable frames #4882

Open

Description

@KirillZiborov

Describe the bug
A skippable frame whose Frame_Size is between 0xFFFFFFF8 and 0xFFFFFFFF is refused with frameParameter_unsupported by ZSTD_findFrameCompressedSize and ZSTD_decompress. ZSTD_decompressStream skips the same frame without error. RFC 8878, section 3.1.2, allows User_Data of up to 2^32-1 bytes.

Cause: readSkippableFrameSize checks that sizeU32 + 8 fits in 32 bits (L594-L595), although it then computes the frame size in size_t. ZSTD_findDecompressedSize, ZSTD_decompressBound and ZSTD_readSkippableFrame go through the same function and refuse the frame too (output below).

To Reproduce
The PoC builds one skippable frame with Frame_Size = 0xFFFFFFF8 (a 4 GiB buffer from calloc).

  1. make -C lib libzstd.a
  2. cc -O2 -Ilib poc.c lib/libzstd.a -o poc && ./poc
  3. Output:
frame of 4294967296 bytes (User_Data 4294967288 bytes)
ZSTD_findFrameCompressedSize: Unsupported frame parameter
ZSTD_decompress: Unsupported frame parameter
ZSTD_decompressStream: 0, read 4294967296 of 4294967296

With -DSIZE=0xFFFFFFF0u, just below the limit, every call accepts the frame.

The same frame passed to the other functions that read a skippable frame's size (ZSTD_STATIC_LINKING_ONLY), on dev and v1.5.7:

ZSTD_findDecompressedSize: ZSTD_CONTENTSIZE_ERROR
ZSTD_decompressBound: ZSTD_CONTENTSIZE_ERROR
ZSTD_readSkippableFrame: Src size is incorrect
poc.c
#include <stdio.h>
#include <stdlib.h>
#include "zstd.h"
#ifndef SIZE
#define SIZE 0xFFFFFFF8u
#endif

int main(void)
{
    size_t len = 8 + (size_t) SIZE;
    unsigned char *f = calloc(len, 1), out[1];
    if (f == NULL) { printf("no memory for %zu bytes\n", len); return 1; }
    f[0] = 0x50; f[1] = 0x2A; f[2] = 0x4D; f[3] = 0x18;                       /* Magic_Number 0x184D2A50 */
    f[4] = (unsigned char) SIZE; f[5] = (unsigned char) (SIZE >> 8);          /* Frame_Size, little-endian */
    f[6] = (unsigned char) (SIZE >> 16); f[7] = (unsigned char) (SIZE >> 24);
    size_t fs = ZSTD_findFrameCompressedSize(f, len);
    size_t d = ZSTD_decompress(out, sizeof out, f, len);
    printf("frame of %zu bytes (User_Data %u bytes)\n", len, SIZE);
    printf("ZSTD_findFrameCompressedSize: %s\n", ZSTD_isError(fs) ? ZSTD_getErrorName(fs) : "accepted");
    printf("ZSTD_decompress: %s\n", ZSTD_isError(d) ? ZSTD_getErrorName(d) : "accepted");
    ZSTD_DStream *zds = ZSTD_createDStream();
    ZSTD_inBuffer in = { f, len, 0 };
    ZSTD_outBuffer o = { out, sizeof out, 0 };
    size_t r = ZSTD_decompressStream(zds, &o, &in);
    printf("ZSTD_decompressStream: %s, read %zu of %zu\n", ZSTD_isError(r) ? ZSTD_getErrorName(r) : "0", in.pos, len);
    ZSTD_freeDStream(zds);
    free(f);
    return 0;
}

Expected behavior
One-shot decoding and the frame-size functions accept the frame, as streaming does: ZSTD_findFrameCompressedSize returns 4294967296, ZSTD_decompress, ZSTD_findDecompressedSize and ZSTD_decompressBound return 0, and ZSTD_readSkippableFrame returns the 4294967288 bytes of User_Data.

Desktop:

  • OS: Ubuntu 24.04.4 LTS
  • Version: dev 49cf517, v1.5.7 and v1.4.9 (the check is the same in all three)
  • Compiler: gcc 13.3.0
  • Flags: default lib/Makefile flags, PoC built with -O2
  • Other relevant hardware specs: x86_64
  • Build system: Makefile

Additional context
Suggested fix: compare in size_t. srcSize >= 8 is checked just before, so neither side wraps, also with a 32-bit size_t. A truncated frame then gives srcSize_wrong, as it already does below the limit.

--- a/lib/decompress/zstd_decompress.c
+++ b/lib/decompress/zstd_decompress.c
@@ -591,12 +591,8 @@ static size_t readSkippableFrameSize(void const* src, size_t srcSize)
     RETURN_ERROR_IF(srcSize < ZSTD_SKIPPABLEHEADERSIZE, srcSize_wrong, "");
 
     sizeU32 = MEM_readLE32((BYTE const*)src + ZSTD_FRAMEIDSIZE);
-    RETURN_ERROR_IF((U32)(sizeU32 + ZSTD_SKIPPABLEHEADERSIZE) < sizeU32,
-                    frameParameter_unsupported, "");
-    {   size_t const skippableSize = skippableHeaderSize + sizeU32;
-        RETURN_ERROR_IF(skippableSize > srcSize, srcSize_wrong, "");
-        return skippableSize;
-    }
+    RETURN_ERROR_IF(sizeU32 > srcSize - skippableHeaderSize, srcSize_wrong, "");
+    return skippableHeaderSize + sizeU32;
 }
 
 /*! ZSTD_readSkippableFrame() :

Found by genproof.com Formal Verification Agent. Reviewed and approved by a human.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions