Skip to content

Run the tests in GitHub Actions - #22

Open
tas50 wants to merge 1 commit into
chef:mainfrom
tas50:ci-github-actions
Open

tas50 wants to merge 1 commit into
chef:mainfrom
tas50:ci-github-actions

Conversation

@tas50

@tas50 tas50 commented Sep 8, 2026

Copy link
Copy Markdown

CI here was .travis.yml, targeting OTP 18.3 through 21.0. Travis stopped serving open source projects in 2021, so nothing has run against this repo since. This replaces it with a GitHub Actions workflow that compiles and runs eunit on OTP 26, 27 and 28 — 26 being what chef-server currently pins, with 27 and 28 covered so regressions surface before that pin moves.

Three things had to be fixed before the suite would run at all.

1. eper blocked the entire test profile

It is a debugging tool, not a test dependency — declared in rebar.config and referenced nowhere in src/ or test/. Its upstream was last pushed in 2018, and prfPrc.erl matches on the float 0.0, which OTP 26 warns about on behalf of OTP 27:

_build/test/lib/eper/src/prfPrc.erl:120: matching on the float 0.0 will no longer
also match -0.0 in OTP 27
===> Compiling _build/test/lib/eper/src/prfPrc.erl failed

warnings_as_errors turns that into a failure, so rebar3 eunit never got as far as running a test. Removed.

2. Most of the suite is not a unit test

opscoderl_ibrowse_test_, multi_request_test and process_leak_test make live requests to twelve third-party hosts and assert HTTP 200 from them:

google.com · www.sun.com · www.oracle.com · www.bbc.co.uk · yaws.hyber.org · www.httpwatch.com · github.com · mail.google.com · jigsaw.w3.org (eighteen times)

As a merge gate that fails whenever one of those is slow, moved or gone. Three already fail on OTP 26, where ssl defaults to verify_peer and those requests supply no CA certs:

{conn_failed,{error,{options,incompatible,[{verify,verify_peer},{cacerts,undefined}]}}}

They are kept, not deleted — run them with OPSCODERL_HTTPC_NETWORK_TESTS=1. By default the workflow runs only the deterministic suites.

3. request_error_test_ only passed because something else ran first

It calls oc_httpc:add_pool/2, which needs pooler running, but its own setup started only ibrowse — pooler was being started by opscoderl_ibrowse_test_'s setup. Skipping the network suites surfaced this immediately as two cancelled tests. Its setup now starts pooler itself, so it stands alone.

Verified

Result
OTP 26 3 tests, 0 failures
OTP 27 3 tests, 0 failures
OTP 28 3 tests, 0 failures
OTP 26 + OPSCODERL_HTTPC_NETWORK_TESTS=1 34 tests, 3 failures (the TLS change above)

Dialyzer is included but not a gate

Four warnings stand on main:

  • Three are lager's parse transform expanding into lager:do_log/10, lager:md/0 and lager_config:get/2, which dialyzer cannot resolve. Move off lager onto OTP logger #21 removes lager and clears all three — I confirmed that separately (4 warnings → 1).
  • One is pooler:time_spec/0: pooler uses that type internally but never -export_types it, so oc_httpc.erl:134's spec refers to something that is not public. plt_extra_apps does not help — I tried. It needs either an export in pooler or a local type here.

Once both are addressed the job should drop continue-on-error and gate.

Also

.gitignore gains _build/ and rebar.lock. Its existing entries (deps/, ebin/, .eunit) are rebar2's, so any rebar3 build currently leaves the tree dirty.

CI here was .travis.yml, targeting OTP 18.3 through 21.0. Travis stopped
serving open source projects in 2021, so nothing has run against this
repo since. This replaces it with a GitHub Actions workflow that
compiles and runs eunit on OTP 26, 27 and 28 -- 26 being what
chef-server currently pins, with 27 and 28 covered so regressions show
up before that pin moves.

Three things had to be fixed before the suite could run at all.

eper blocked the test profile. It is a debugging tool, not a test
dependency: it is declared in rebar.config and referenced nowhere in
src/ or test/. Its upstream was last pushed in 2018, and prfPrc.erl
matches on the float 0.0, which OTP 26 warns about on behalf of OTP 27.
warnings_as_errors turns that into a failure, so `rebar3 eunit` did not
get as far as running a test. Removed.

Most of the suite is not a unit test. opscoderl_ibrowse_test_,
multi_request_test and process_leak_test make live requests to a dozen
third-party hosts -- google.com, www.sun.com, yaws.hyber.org,
www.httpwatch.com, jigsaw.w3.org (eighteen times) and others -- and
assert HTTP 200 from them. As a merge gate that fails whenever one of
those is slow, moved or gone, and three already fail on OTP 26 because
ssl now defaults to verify_peer and those requests supply no CA certs.
They are kept and can be run by setting
OPSCODERL_HTTPC_NETWORK_TESTS=1; by default the workflow runs only the
deterministic suites.

request_error_test_ only passed because something else ran first. It
calls oc_httpc:add_pool/2, which needs pooler running, but its setup
started only ibrowse -- pooler was started by opscoderl_ibrowse_test_'s
setup. Skipping the network suites surfaced that immediately as two
cancelled tests. Its setup now starts pooler itself.

Verified in containers: 3 tests, 0 failures on OTP 26, 27 and 28, and
34 tests with OPSCODERL_HTTPC_NETWORK_TESTS=1 (3 failing, on the TLS
change described above).

Dialyzer is included but not a gate. Four warnings stand on main: three
are lager's parse transform expanding into lager:do_log/10, lager:md/0
and lager_config:get/2, which dialyzer cannot resolve -- removing lager
clears all three -- and one is pooler:time_spec/0, a type pooler uses
internally but never exports, so oc_httpc's spec refers to something
that is not public. Once both are addressed the job should drop
continue-on-error and gate.

.gitignore gains _build/ and rebar.lock; its existing entries (deps/,
ebin/, .eunit) are rebar2's, so a rebar3 build leaves the tree dirty.

Signed-off-by: Tim Smith <tim@mondoo.com>
@tas50

tas50 commented Sep 8, 2026

Copy link
Copy Markdown
Author

Confirmed the workflow actually executes, rather than only that the commands work locally.

The run on chef/opscoderl_httpc shows action_required — GitHub holds fork workflows pending maintainer approval — so I opened the identical PR against my own fork to get a real execution:

Job Result
OTP 26 ✅ success
OTP 27 ✅ success
OTP 28 ✅ success
Dialyzer (informational) ❌ failure
Overall run ✅ success

That is the intended behaviour: dialyzer reports its four known warnings without failing the run, because of continue-on-error: true. When #21 lands (clearing the three lager ones) and pooler:time_spec/0 is resolved, that job should drop continue-on-error and become a gate.

Approving the workflow run on this PR will reproduce the same result here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant