Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,12 @@ jobs:
cargo check -p "$pkg" --all-targets || exit 1
done

# Dry-runs the publish without compiling, which catches dependency cycles between
# workspace crates, versions that do not resolve on crates.io, and missing metadata.
# publish.yml runs the full dry-run with compilation on release candidate tags.
- name: Check each crate can be published
run: cargo publish --workspace --all-features --dry-run --no-verify

# Verifies the core iceberg crate compiles without default features, ensuring
# optional functionality is properly gated behind feature flags.
build_with_no_default_features:
Expand Down
7 changes: 7 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,13 @@ jobs:
with:
rust-version: ${{ steps.get-msrv.outputs.msrv }}

# Packages and builds every crate as it would be published, without uploading,
# so publish failures are caught during the release candidate process.
- name: Dry-run publish for release candidates
if: ${{ startsWith(github.ref, 'refs/tags/') && contains(github.ref, '-') }}
run: cargo publish --workspace --all-features --dry-run
shell: bash

- uses: rust-lang/crates-io-auth-action@c6f97d42243bad5fab37ca0427f495c86d5b1a18 # v1.0.5
id: auth

Expand Down
6 changes: 4 additions & 2 deletions crates/property-macro/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ license = { workspace = true }
repository = { workspace = true }

categories = ["database"]
description = "Property derive macro for Apache Iceberg Rust"
description = "Apache Iceberg Rust property macros, internal to the iceberg crate"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i feel like its a good idea to call this out as "internal", but happy to revert

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I agree to add internal note to description.

keywords = ["iceberg"]

[lib]
Expand All @@ -40,7 +40,9 @@ quote = { workspace = true }
syn = { workspace = true, features = ["full"] }

[dev-dependencies]
iceberg = { workspace = true }
# Path-only so cargo strips it when packaging; a versioned dev-dependency would
# create a publish cycle, since `iceberg` depends on this crate.
iceberg = { path = "../iceberg" }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the main fix

serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
trybuild = { workspace = true }
Expand Down
5 changes: 5 additions & 0 deletions crates/property-macro/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@

# Iceberg property macros

This crate contains the property macros used internally by
[`iceberg`](https://crates.io/crates/iceberg). It is published only because
`iceberg` depends on it and has no API stability guarantees, so depend on
`iceberg` rather than on this crate.

## `Properties` derive macro

`#[derive(Properties)]` parses an owned typed struct from a flat
Expand Down
6 changes: 5 additions & 1 deletion dev/release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Common options:
dev/release/create_rc.sh 0.9.1 2 --release_ref <commit-ish>
dev/release/create_rc.sh 0.9.1 2 --create_rc_tag 0 --sign 0
dev/release/create_rc.sh 0.9.1 2 --upload_svn 1
dev/release/create_rc.sh 0.9.1 2 --check_headers 0 --check_deps 0
dev/release/create_rc.sh 0.9.1 2 --check_headers 0 --check_deps 0 --check_publish 0
```

Defaults:
Expand All @@ -52,6 +52,7 @@ Defaults:
- `--create_rc_tag 1`: create the signed annotated RC tag as the final release step.
- `--check_headers 1`: check Apache license headers against the source archive.
- `--check_deps 1`: run dependency license checks before artifact creation.
- `--check_publish 1`: dry-run publishing every crate to crates.io before artifact creation.
- `--sign 1`: create and verify the detached GPG signature.
- `--upload_svn 0`: upload RC artifacts to the ASF dev dist SVN repository.
- `--svn_dist_url https://dist.apache.org/repos/dist/dev/iceberg`: SVN directory URL where the RC artifact directory will be uploaded.
Expand All @@ -64,6 +65,9 @@ Defaults:
cargo install --locked cargo-deny
```

`--check_publish 1` runs `cargo publish --workspace --dry-run`, which packages and compiles every crate and needs network access to crates.io.
Pass `--check_publish 0` to skip it when offline or when iterating on the script.

## Verify an RC

```shell
Expand Down
24 changes: 24 additions & 0 deletions dev/release/create_rc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ Options:
Whether to run the dependency license check before creating artifacts.
Default: 1

--check_publish <0|1>
Whether to dry-run publishing every crate to crates.io before creating artifacts.
Default: 1

--sign <0|1>
Whether to create and verify the detached GPG signature for the source archive.
Default: 1
Expand Down Expand Up @@ -227,6 +231,12 @@ parse_args() {
CHECK_DEPS="$2"
shift 2
;;
--check_publish | --check-publish)
require_option_value "$1" "${2:-}"
validate_bool_option "$1" "$2"
CHECK_PUBLISH="$2"
shift 2
;;
--sign)
require_option_value "$1" "${2:-}"
validate_bool_option "$1" "$2"
Expand Down Expand Up @@ -315,6 +325,13 @@ check_dependency_licenses() {
)
}

# Packages and builds every crate as `cargo publish` would, without uploading,
# so crates that cannot be published fail RC creation.
check_crates_publishable() {
require_command cargo
cargo publish --workspace --all-features --dry-run --manifest-path "${REPO_ROOT}/Cargo.toml"
}

prepare_output_directory() {
rm -rf "${RC_DIR}"
mkdir -p "${RC_DIR}"
Expand Down Expand Up @@ -500,6 +517,7 @@ RELEASE_DIST_DIR="dist"
CREATE_RC_TAG="1"
CHECK_HEADERS="1"
CHECK_DEPS="1"
CHECK_PUBLISH="1"
SIGN_ARCHIVE="1"
UPLOAD_SVN="0"
SVN_DIST_URL="https://dist.apache.org/repos/dist/dev/iceberg"
Expand All @@ -523,6 +541,12 @@ else
skip_step "Check dependency licenses"
fi

if enabled "${CHECK_PUBLISH}"; then
run_step "Check crates can be published" check_crates_publishable
else
skip_step "Check crates can be published"
fi

run_step "Prepare output directory ${RC_DIR}" prepare_output_directory
run_step "Create source archive ${ARCHIVE_FILE_NAME}" create_source_archive

Expand Down
6 changes: 5 additions & 1 deletion website/src/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@ Useful options include:
- `--create_rc_tag 1`: create the signed annotated RC tag as the final release step.
- `--check_headers 1`: check Apache license headers against the source archive.
- `--check_deps 1`: run dependency license checks before artifact creation.
- `--check_publish 1`: dry-run publishing every crate to crates.io before artifact creation.
- `--sign 1`: create and verify the detached GPG signature.
- `--upload_svn 0`: upload RC artifacts to the ASF dev dist SVN repository.
- `--svn_dist_url https://dist.apache.org/repos/dist/dev/iceberg`: SVN directory URL where the RC artifact directory will be uploaded.
Expand All @@ -214,7 +215,10 @@ This script creates:
- SHA-512 checksum: `apache-iceberg-rust-${iceberg_version}.tar.gz.sha512`
- Signed annotated RC tag: `v${iceberg_version}-rc.${rc}`

The script checks license headers against the generated source archive, not the live Git worktree. If enabled, SVN upload runs after local artifact verification and before RC tag creation. The script creates the signed RC tag as the final release step, then prints a draft VOTE email for `dev@iceberg.apache.org`.
The script runs a number of verifications.
It performs a dry-run packaging step and builds every crate as the final release would, so a crate that cannot be published fails RC creation.
It also checks license headers against the generated source archive.
If enabled, SVN upload runs after local artifact verification and before RC tag creation. The script creates the signed RC tag as the final release step, then prints a draft VOTE email for `dev@iceberg.apache.org`.

To upload artifacts to ASF dev dist as part of RC creation, pass:

Expand Down
Loading