Skip to content

Zlib reserving twice the needed memory for inflate and inflateSync, kept in RSS until process exits. #44750

@nitram-work

Description

@nitram-work

Version

v18.9.0 and v16.16.0

Platform

Linux 5.15.0-48-generic #54~20.04.1-Ubuntu SMP Thu Sep 1 16:17:26 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

zlib

What steps will reproduce the bug?

In the following zip file it's a copy of this JS code and the binary file needed to consistently cause the issue.
IMPORTANT: Needs to be called with node --expose-gc

inflate-leak.zip

const fs = require("fs");
const zlib = require("zlib");

const inMB = (b) => (b / 1024 / 1024).toFixed(3) + " MB";

const logMem = async (when) => {
  console.log('-------' + when.padEnd(70, '-'));
  global.gc();
  await new Promise((resolve) => setTimeout(resolve, 1000));
  const mu = process.memoryUsage();
  for (let k in mu) {
    console.log(`${k.padStart(20)} : ${inMB(mu[k])}`);
  }
  await new Promise((resolve) => setTimeout(resolve, 1000));
};



const main = async () => {
  await logMem("initial");
  let deflated = fs.readFileSync("inflate-leak.bin");
  await logMem(`deflated ${inMB(deflated.length)}`);

  let inflated = zlib.inflateSync(deflated);
  //let inflated = await new Promise(resolve => zlib.inflate(deflated, (err, buf) => resolve(buf)));
  await logMem(`inflated size ${inMB(inflated.length)}`);
  inflated = null;
  await logMem("inflated=null");
  deflated = null;
  await logMem("deflated=null");

  for(let i = 0; i<100; i++)
    global.gc();
  
  await new Promise((resolve) => setTimeout(resolve, 10_000));
  await logMem("after 10 seconds");
};

main().then();

How often does it reproduce? Is there a required condition?

Every time for the attached binary file.

What is the expected behavior?

Inflate reserving only the needed size (~1.6GB) instead of double (~3.2GB).
RSS lowered reflecting array buffers/external lowered size after garbage collection.

What do you see instead?

After inflate or inflateSync RSS stays with an extra 1666MB

-------initial---------------------------------------------------------------
                 rss : 40.477 MB
           heapTotal : 6.746 MB
            heapUsed : 4.075 MB
            external : 0.364 MB
        arrayBuffers : 0.016 MB
-------deflated 2.116 MB-----------------------------------------------------
                 rss : 42.492 MB
           heapTotal : 5.996 MB
            heapUsed : 4.312 MB
            external : 2.479 MB
        arrayBuffers : 2.132 MB
-------inflated size 1598.928 MB---------------------------------------------
                 rss : 3269.332 MB
           heapTotal : 10.246 MB
            heapUsed : 4.249 MB
            external : 3200.344 MB
        arrayBuffers : 1601.060 MB
-------inflated=null---------------------------------------------------------
                 rss : 1669.563 MB
           heapTotal : 9.246 MB
            heapUsed : 4.234 MB
            external : 1601.407 MB
        arrayBuffers : 2.132 MB
-------deflated=null---------------------------------------------------------
                 rss : 1666.797 MB
           heapTotal : 6.246 MB
            heapUsed : 4.663 MB
            external : 2.479 MB
        arrayBuffers : 0.016 MB
-------after 10 seconds------------------------------------------------------
                 rss : 1666.582 MB
           heapTotal : 5.996 MB
            heapUsed : 4.281 MB
            external : 0.363 MB
        arrayBuffers : 0.016 MB

Additional information

The binary file is the deflated data chunk of a 400 mega pixel PNG

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions