build fails on macOS due to "implicit declaration of function 'lseek' is invalid in C99"

(pip install blosc also results in the same errors; the following is my attempt to build from source)

macOS version 10.15.6 (19G73)
Python 3.8.5

My Python has among others
cmake==3.18.0
scikit-build==0.11.1

brew install c-blosc
python setup.py build_clib
BLOSC_DIR=/usr/local/Cellar/c-blosc/ python setup.py build_ext --inplace

output:

/Users/xian/pandas-dev-env/lib/python3.8/site-packages/setuptools/distutils_patch.py:25: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
  warnings.warn(
/Users/xian/pandas-dev-env/lib/python3.8/site-packages/setuptools/distutils_patch.py:25: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
  warnings.warn(
/Users/xian/pandas-dev-env/lib/python3.8/site-packages/setuptools/dist.py:458: UserWarning: Normalizing '1.9.2-dev0' to '1.9.2.dev0'
  warnings.warn(tmpl.format(**locals()))


--------------------------------------------------------------------------------
-- Trying "Ninja" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
Not searching for unused variables given on the command line.
CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
-- Configuring incomplete, errors occurred!
See also "/Users/xian/Code/python-blosc/_cmake_test_compile/build/CMakeFiles/CMakeOutput.log".
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Ninja" generator - failure
--------------------------------------------------------------------------------



--------------------------------------------------------------------------------
-- Trying "Unix Makefiles" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
Not searching for unused variables given on the command line.
-- The C compiler identification is AppleClang 12.0.0.12000026
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is AppleClang 12.0.0.12000026
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/xian/Code/python-blosc/_cmake_test_compile/build
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Unix Makefiles" generator - success
--------------------------------------------------------------------------------

Configuring Project
  Working directory:
    /Users/xian/Code/python-blosc/_skbuild/macosx-10.15-x86_64-3.8/cmake-build
  Command:
    cmake /Users/xian/Code/python-blosc -G 'Unix Makefiles' -DCMAKE_INSTALL_PREFIX:PATH=/Users/xian/Code/python-blosc/_skbuild/macosx-10.15-x86_64-3.8/cmake-install -DPYTHON_EXECUTABLE:FILEPATH=/Users/xian/pandas-dev-env/bin/python -DPYTHON_VERSION_STRING:STRING=3.8.5 -DPYTHON_INCLUDE_DIR:PATH=/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8 -DPYTHON_LIBRARY:FILEPATH=/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/libpython3.8.dylib -DSKBUILD:BOOL=TRUE -DCMAKE_MODULE_PATH:PATH=/Users/xian/pandas-dev-env/lib/python3.8/site-packages/skbuild/resources/cmake -DBLOSC_DIR:PATH=/usr/local/Cellar/c-blosc/ -DDEACTIVATE_SSE2:BOOL=OFF -DDEACTIVATE_AVX2:BOOL=OFF -DDEACTIVATE_LZ4:BOOL=OFF -DDEACTIVATE_SNAPPY:BOOL=ON -DDEACTIVATE_ZLIB:BOOL=OFF -DDEACTIVATE_ZSTD:BOOL=OFF -DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=10.15 -DCMAKE_OSX_ARCHITECTURES:STRING=x86_64

-- The C compiler identification is AppleClang 12.0.0.12000026
-- The CXX compiler identification is AppleClang 12.0.0.12000026
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /Users/xian/pandas-dev-env/bin/python (found version "3.8.5") 
-- Found PythonLibs: /usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/libpython3.8.dylib (found version "3.8.5") 
Configuring for Blosc version: 1.18.1
-- Using LZ4 internal sources.
-- Using zlib internal sources.
-- Using ZSTD internal sources.
-- Building for system processor x86_64
-- Detected that BLOSC is used a subproject.
-- Adding run-time support for SSE2
-- Adding run-time support for AVX2
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Performing Test Weak Link MODULE -> SHARED (gnu_ld_ignore) - Failed
-- Performing Test Weak Link MODULE -> SHARED (osx_dynamic_lookup) - Success
_modinit_prefix:PyInit_
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    BLOSC_DIR
    SKBUILD


-- Build files have been written to: /Users/xian/Code/python-blosc/_skbuild/macosx-10.15-x86_64-3.8/cmake-build
Scanning dependencies of target blosc_static
[  1%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/blosc.c.o
[  3%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/blosclz.c.o
[  5%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/fastcopy.c.o
[  7%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/shuffle-generic.c.o
[  9%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/bitshuffle-generic.c.o
[ 11%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/shuffle-sse2.c.o
[ 12%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/bitshuffle-sse2.c.o
[ 14%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/shuffle-avx2.c.o
[ 16%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/bitshuffle-avx2.c.o
[ 18%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/shuffle.c.o
[ 20%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/__/internal-complibs/lz4-1.9.2/lz4.c.o
[ 22%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/__/internal-complibs/lz4-1.9.2/lz4hc.c.o
[ 24%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/__/internal-complibs/zlib-1.2.8/adler32.c.o
[ 25%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/__/internal-complibs/zlib-1.2.8/compress.c.o
[ 27%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/__/internal-complibs/zlib-1.2.8/crc32.c.o
[ 29%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/__/internal-complibs/zlib-1.2.8/deflate.c.o
[ 31%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/__/internal-complibs/zlib-1.2.8/gzclose.c.o
[ 33%] Building C object c-blosc/blosc/CMakeFiles/blosc_static.dir/__/internal-complibs/zlib-1.2.8/gzlib.c.o
/Users/xian/Code/python-blosc/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:256:24: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        state->start = LSEEK(state->fd, 0, SEEK_CUR);
                       ^
/Users/xian/Code/python-blosc/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
#  define LSEEK lseek
                ^
/Users/xian/Code/python-blosc/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:256:24: note: did you mean 'fseek'?
/Users/xian/Code/python-blosc/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
#  define LSEEK lseek
                ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:162:6: note: 'fseek' declared here
int      fseek(FILE *, long, int);
         ^
/Users/xian/Code/python-blosc/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:355:9: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    if (LSEEK(state->fd, state->start, SEEK_SET) == -1)
        ^
/Users/xian/Code/python-blosc/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
#  define LSEEK lseek
                ^
/Users/xian/Code/python-blosc/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:396:15: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        ret = LSEEK(state->fd, offset - state->x.have, SEEK_CUR);
              ^
/Users/xian/Code/python-blosc/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
#  define LSEEK lseek
                ^
/Users/xian/Code/python-blosc/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:492:14: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    offset = LSEEK(state->fd, 0, SEEK_CUR);
             ^
/Users/xian/Code/python-blosc/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
#  define LSEEK lseek
                ^
4 errors generated.
make[2]: *** [c-blosc/blosc/CMakeFiles/blosc_static.dir/__/internal-complibs/zlib-1.2.8/gzlib.c.o] Error 1
make[1]: *** [c-blosc/blosc/CMakeFiles/blosc_static.dir/all] Error 2
make: *** [all] Error 2
Traceback (most recent call last):
  File "/Users/xian/pandas-dev-env/lib/python3.8/site-packages/skbuild/setuptools_wrap.py", line 589, in setup
    cmkr.make(make_args, env=env)
  File "/Users/xian/pandas-dev-env/lib/python3.8/site-packages/skbuild/cmaker.py", line 496, in make
    raise SKBuildError(

An error occurred while building with CMake.
  Command:
    cmake --build . --target install --config Release --
  Source directory:
    /Users/xian/Code/python-blosc
  Working directory:
    /Users/xian/Code/python-blosc/_skbuild/macosx-10.15-x86_64-3.8/cmake-build
Please see CMake's output for more information.

solved with

diff --git a/c-blosc/internal-complibs/zlib-1.2.8/gzguts.h b/c-blosc/internal-complibs/zlib-1.2.8/gzguts.h
index d87659d..fb91985 100644
--- a/c-blosc/internal-complibs/zlib-1.2.8/gzguts.h
+++ b/c-blosc/internal-complibs/zlib-1.2.8/gzguts.h
@@ -3,6 +3,8 @@
  * For conditions of distribution and use, see copyright notice in zlib.h
  */
 
+#include <unistd.h>
+
 #ifdef _LARGEFILE64_SOURCE
 #  ifndef _LARGEFILE_SOURCE
 #    define _LARGEFILE_SOURCE 1

BLOSC_DIR=/usr/local/Cellar/c-blosc/ python setup.py build_ext --inplace
python setup.py install

Hmmm, I cannot reproduce this. My MacOSX box is 10.15.5 and python 3.7.6. I wonder whether a new version of zlib can fix your problem.

@xytxytxyt

@xytxytxyt Same issue, same resolution. Any chance we can patch this soon?

I have just updated to latest Mac OSX 10.15.6, but I can still compile the beast with Python 3.7.6. I suppose this is a Python 3.8 thing. We will try to appy your patch to C-Blosc sources and propagate this back to python-blosc soon.

Understood, thanks. For the time being, I'll just build from source with your recommendations above

Add a header in zlib-1.2.8 sources. Addresses Blosc/python-blosc#229

Protect header inclusion on Win. Addresses Blosc/python-blosc#229

python-blosc 1.9.2 has been released and fixes this. Closing.

@FrancescAlted

Thank you so much @FrancescAlted !!

Fix Continuous Integration build for Max by including unistd.h

Update python-blosc to version 1.11.4 / rev 27 via SR 1346169

Navigation Menu

Repository navigation

build fails on macOS due to "implicit declaration of function 'lseek' is invalid in C99" #229

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

build fails on macOS due to "implicit declaration of function 'lseek' is invalid in C99" #229

Description

Activity

xytxytxyt commented on Aug 19, 2020

FrancescAlted commented on Aug 21, 2020

tsu-shiuan commented on Sep 3, 2020

FrancescAlted commented on Sep 3, 2020

tsu-shiuan commented on Sep 4, 2020

FrancescAlted commented on Sep 9, 2020

tsu-shiuan commented on Sep 14, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions