Skip to content

Tests fail in 1-core VM #4410

Closed
Closed

Description

@bmwiedemann

What happened:
running the openSUSE python-distributed package build+test in a 1-core VM failed 2 tests.

distributed/tests/test_diskutils.py .....FF
test_workspace_concurrency
test_workspace_concurrency_intense

What you expected to happen:
all tests should succeed. If there are tests that do not make sense on 1 core, cpu_count() should be used to skip them.

Minimal Complete Verifiable Example:

taskset 1 pytest-3.8 distributed/tests/test_diskutils.py

Anything else we need to know?:

Environment:

  • Dask version: 2.25.0
  • Python version: 3.8.6
  • Operating System: openSUSE-Tumbleweed 20210104
  • Install method: source / rpm .spec

This bug was found while working on reproducible builds for openSUSE.

Activity

  1. quasiben commented on Jan 7, 2021

    @quasiben
    Member

    What were the failures of the test ?

    A few additional questions:

    • Are you the trying to build official openSuse packages for Dask/Distributed ?
    • Is 2.25.0 particularly important? If not, I would suggest upgrading to latest: 2020.12.0
  2. bmwiedemann commented on Jan 7, 2021

    @bmwiedemann
    Author

    I thought, it would be easy to reproduce, so I skipped the full log before. Now I uploaded for you https://rb.zq1.de/temp/python-distributed-j1-build-log.txt

    Yes, I try to rebuild the official openSUSE packages.
    On Debian or openSUSE you can do:

    osc co openSUSE:Factory/python-distributed && cd $_
    taskset 1 osc build --noservice -j1

    I tried to switch to 2020.12.0 but it seems to not work with the other versions we have in openSUSE Tumbleweed:

     distributed/client.py:32: in <module>
         from dask.utils import ensure_dict, format_bytes, funcname, stringify
     E   ImportError: cannot import name 'stringify' from 'dask.utils' (/usr/lib/python3.8/site-packages/dask/utils.py)
    
  3. jacobtomlinson commented on Jan 7, 2021

    @jacobtomlinson
    Member

    If you switch to 2020.12.0 you probably also need dask at the same version.

  4. jrbourbeau commented on Jan 7, 2021

    @jrbourbeau
    Member

    If you switch to 2020.12.0 you probably also need dask at the same version.

    Correct, the stringify function in the ImportError was added to Dask in 2020.12.0

  5. bmwiedemann commented on Jan 8, 2021

    @bmwiedemann
    Author

    also interesting:
    https://build.opensuse.org/package/live_build_log/home:bmwiedemann:reproducible:dask/python-dask:test/openSUSE_Tumbleweed/x86_64 fails:

    [  136s] =================================== FAILURES ===================================
    [  136s] ______________________________ test_deterministic ______________________________
    [  136s] 
    [  136s]     def test_deterministic():
    [  136s]         pytest.importorskip("mimesis")
    [  136s]     
    [  136s]         b = dask.datasets.make_people(seed=123)
    [  136s] >       assert b.take(1)[0]["name"] == ("Leandro", "Orr")
    [  136s] E       AssertionError: assert ('Luke', 'Ellis') == ('Leandro', 'Orr')
    

    could that be influenced by our PYTHONHASHSEED env?

  6. jacobtomlinson commented on Jan 8, 2021

    @jacobtomlinson
    Member

    could that be influenced by our PYTHONHASHSEED env?

    Probably

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions