Release 2022.10.1

@jrbourbeau


Release version	2022.10.1
Planned release date	2022-10-28
Status	On-track
Release manager	@jrbourbeau

Best effort

Try to close before the release but will not block the release

Blocker

Issues that would cause us to block and postpone the release if not fixed

Dashboard plot sizing issue with bokeh=3 distributed#7173 (noting that this is no longer a blocker, even though the issue is still open as we've added temporary workarounds for bokeh=3 compatibility -- see the PRs linked in Release 2022.10.1 #283 (comment))
...

Comments

cc @ian-r-rose @fjetter @quasiben @jakirkham

Digging a bit with the CLI issue..

After installing dask and distributed with conda-forge

If I run python -m dask...

$ python -m dask

shows what we want:

Usage: python -m dask [OPTIONS] COMMAND [ARGS]...

  Dask command line interface.

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  docs       Open Dask documentation (https://docs.dask.org/) in a web browser.
  info       Information about your dask installation.
  scheduler  Launch a distributed scheduler.
  ssh        Launch a distributed cluster over SSH.
  worker     Launch a distributed worker attached to an existing SCHEDULER.

but

$ dask

shows

Usage: dask [OPTIONS] COMMAND [ARGS]...

  Dask command line interface.

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  docs  Open Dask documentation (https://docs.dask.org/) in a web browser.
  info  Information about your dask installation.

$ python -m dask is running dask.__main__:main while $ dask is running dask.cli:cli. (note the pip(setuptools) installation uses dask.__main__:main). I think the conda-forge entry point should be dask.__main__:main

~~Opened a PR conda-forge/dask-feedstock#200~~ fixed in dask-core-feedstock!

Thanks all for helping dig into the CLI issues! 🙏

Given the pain around that, would like to propose hardening the Click requirement in Dask going forward ( dask/dask#9595 ). Ideally would like to include that in this release

I've opened this issue around requiring python<3.11 dask/dask#9605. If folks have thoughts on the topic I'd appreciate feedback on that issue.

@fjetter

We haven't quite gotten dask/distributed#5648 over the finish line yet, so I'm going to push up PRs to put an upper bound of bokeh < 3 for dask and distributed given that we know there are compatibility issues with the bokeh=3 release scheduled to come out later today (cc @fjetter @crusaderky)

@gjoseph92

@gjoseph92 @fjetter are we turning on task queuing on the scheduler by default in this release?

No, it won't be ready in time. See dask/distributed#7213.

bokeh < 3 PRs:

@jrbourbeau

@jrbourbeau not critical, but could be good to get dask/distributed#7224 in if CI passes and you have time to review.

@gjoseph92

Ah, sorry @gjoseph92 I need to start pushing out the release. Unfortunately I think that one will have to get in in the next release

No worries, queuing is still behind a feature flag so it's not too big of a deal.

2022.10.1 is out on PyPI and conda-forge PRs are open:

@jacobtomlinson

Alright, PyPI and conda-forge packages are released.

We ran into a problem building the Python 3.8 dask-notebook image over in dask-docker (xref dask/dask-docker#255) due to an issue in the underlying jupyter/base-notebook notebook (xref jupyter/docker-stacks#1810). I've attempted a couple of possible fixes, but haven't gotten those build to work yet. I've proposed a temporary workaround in dask/dask-docker#256 where we drop Python 3.8 builds. This is a possible workaround, but admittedly feels a bit heavy handed. I'm on PTO next week -- @jacobtomlinson @jakirkham @charlesbluca could I ask you to handle docker images for this release by either finding an actual fix, or by merging the PR that temporarily drops Python 3.8 builds?

FWIW commented on the upstream (Jupyter) issue.

Heads up: dask/dask#9609 Does anyone here have time to investigate what happened here with Bokeh? It appears that we're affecting Airflow negatively.

…

On Sat, Oct 29, 2022 at 3:53 AM jakirkham ***@***.***> wrote: FWIW commented on the upstream (Jupyter) issue. — Reply to this email directly, view it on GitHub <#283 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTFYPEGQJAQRJA6DSWLWFTQYXANCNFSM6AAAAAAROE5LGY> . You are receiving this because you commented.Message ID: ***@***.***>

Pushing out a hotfix #284

If there is anything I should know folks should jump in very soon.

@gjoseph92

There isn't anything I'm aware of that would prevent a hotfix, though dask/distributed#7224 is a non-trivial change. Perhaps @gjoseph92 can comment on whether it's likely to be an issue (I suspect he'd be fine with it)

@gjoseph92

I think @gjoseph92 is comfortable including that change has he asked for it to be included in the 2022.10.1 release, but it just didn't make it in time (xref #283 (comment))

EDIT: queuing is also still behind a feature flag, so I think it's safe to move forward with that change included

@gjoseph92

I've already chatted with @gjoseph92 and he confirmed.

I've yanked this release from PyPI. Can someone (maybe @jakirkham) help with yanking from conda-forge channels?

could I ask you to handle docker images for this release by either finding an actual fix, or by merging the PR that temporarily drops Python 3.8 builds?

Any opposition to unblocking the 3.8 Docker builds locally with dask/dask-docker#257 until the upstream issue is resolved? This would involve patching the impacted Jupyter dockerfile, which could get hairy in the long run, but it doesn't seem like CI is running very often on dask-docker for this to be an issue.

Would follow these steps for marking packages broken in conda-forge. Unfortunately I don't have bandwidth for this atm

@jakirkham

Thanks @jakirkham! Opened up conda-forge/admin-requests#504 to request the packages be marked as broken

As to Jupyter docker-stacks, outlined a path forward in comment ( jupyter/docker-stacks#1810 (comment) ). If anyone has bandwidth to try, should be a couple line change

Updating this thread, Charles' workaround was merged yesterday. Mamba released 1.0.0 early today. This subsequently fixed Jupyter docker-stacks ( jupyter/docker-stacks#1810 (comment) ). With that fixed, Charles reverted the workaround ( dask/dask-docker#260 ).

Edit: Also merged Charles PR yesterday to mark conda-forge packages broken.

Sounds like there is nothing more to do here. So going to close this out, but please feel free to reopen if I've missed something.

Thanks all! 🙏

Navigation Menu

Repository navigation

Release 2022.10.1 #283

Best effort

Blocker

Comments

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Release 2022.10.1 #283

Description

Best effort

Blocker

Comments

Activity

douglasdavis commented on Oct 25, 2022

douglasdavis commented on Oct 25, 2022

jakirkham commented on Oct 25, 2022

jrbourbeau commented on Oct 27, 2022

jrbourbeau commented on Oct 28, 2022

mrocklin commented on Oct 28, 2022

gjoseph92 commented on Oct 28, 2022

jrbourbeau commented on Oct 28, 2022

gjoseph92 commented on Oct 28, 2022

jrbourbeau commented on Oct 28, 2022

gjoseph92 commented on Oct 28, 2022

jrbourbeau commented on Oct 28, 2022

jrbourbeau commented on Oct 29, 2022

jakirkham commented on Oct 29, 2022

mrocklin commented on Oct 31, 2022

mrocklin commented on Oct 31, 2022

ian-r-rose commented on Oct 31, 2022

jrbourbeau commented on Oct 31, 2022

mrocklin commented on Oct 31, 2022

charlesbluca commented on Oct 31, 2022

jakirkham commented on Oct 31, 2022

charlesbluca commented on Oct 31, 2022

jakirkham commented on Oct 31, 2022

jakirkham commented on Nov 2, 2022

jakirkham commented on Nov 2, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions