Test for Disentangled Spatiotemporal SSM - #2128
Closed
nawazishkhan1-nk wants to merge 5 commits into
Closed
Conversation
rebase with master and conflict fix
nawazishkhan1-nk
force-pushed
the
disentangled_test
branch
from
August 2, 2023 19:19
cefa8b9 to
af520e3
Compare
akenmorris
self-requested a review
August 4, 2023 16:36
akenmorris
requested changes
Aug 4, 2023
Contributor
There was a problem hiding this comment.
Looks good, but this test is much too slow. It looks like it takes a full hour on the github action runners. make test on windows went from 20 minutes to 1:20. The OptimizeTests need to be very fast, not just for GitHub Actions but also so that developers can run them frequently when modifying and refactoring code. Also I saw the following warnings:
[2023-08-04 10:32:20.793] [warning] Unknown Optimization parameter: adaptivity_mode
[2023-08-04 10:32:20.793] [warning] Unknown Optimization parameter: domain_type
[2023-08-04 10:32:20.793] [warning] Unknown Optimization parameter: domains_per_shape
[2023-08-04 10:32:20.793] [warning] Unknown Optimization parameter: recompute_regularization_interval
[2023-08-04 10:32:20.793] [warning] Unknown Optimization parameter: use_statistics_in_init
I'd suggest converting this to swproj and removing any unknown params. The swproj files are more git-friendly such that we can see changes/diffs easily as they are plaintext.
Contributor
|
@nawazishkhan1-nk , any update on this? Can we get a test that runs in <1 minute? |
nawazishkhan1-nk
marked this pull request as draft
August 13, 2024 22:44
akenmorris
added a commit
that referenced
this pull request
Aug 9, 2026
The test added in #2128 took over an hour: 8 subjects x 8 time points at 512 particles with 1000 iterations per split. It now runs 128 particles for 300 iterations and finishes in about 11 seconds while reaching the same model quality as the original. Instead of asserting absolute eigenvalues, the test uses the structure of the fixture: each subject is an ellipsoid whose X radius is fixed per subject and whose Y radius follows a trajectory shared by every subject, so within a time point the subjects differ by a single linear mode. The first PCA mode explaining nearly all of the across-subject variance is therefore a direct measure of correspondence, and it is scale free. It reaches ~0.99 here and falls below 0.2 when correspondence is lost. The per-subject objective matrix was also sized by the number of subjects rather than the number of time points, so any dataset with more time points than subjects aborted. The original fixture is 8x8, which hid it. Fixed, and covered by a second test with a deliberately unequal dataset. The project fixture is now a .swproj rather than a binary .xlsx, and the 128 checked-in particle files that the run regenerates anyway have been dropped. Co-authored-by: Nawazish Khan <nawazishkhan1.nk@gmail.com>
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For #2122