Repository navigation
Conversation
jesperhodge
left a comment
There was a problem hiding this comment.
Great work. This is very solid. The decision to modify queryset is the right one. I'm waiting for the AI code review to come back, in the meantime I identified a few little things.
In terms of the description: it looks pretty good, especially the explanation about why queryset was modified and not the default object manager. I have not managed to read the whole description as it's really long, and I don't think Axim will have time - could you try to shorten the description to something like a quarter of the size? Also a few things at the beginning are a bit hard to understand, but it's not bad. I think after shortening it, you can just run the /clean-up-text skill on what you have and the description should be pretty good.
| assert response.status_code == status.HTTP_200_OK | ||
| assert response.data["read_only"] is True | ||
|
|
||
| def test_detail_taxonomy_archived_404(self) -> None: |
There was a problem hiding this comment.
| def test_detail_taxonomy_archived_404(self) -> None: | |
| def test_get_archived_taxonomy_responds_404(self) -> None: |
| assert response.data["name"] == "new name" | ||
| assert response.data["read_only"] == (not read_only) | ||
|
|
||
| def test_update_taxonomy_archived(self): |
There was a problem hiding this comment.
| def test_update_taxonomy_archived(self): | |
| def test_update_archived_taxonomy_is_400(self): |
| assert response.status_code == status.HTTP_200_OK | ||
| assert response.data["read_only"] == (not read_only) | ||
|
|
||
| def test_patch_taxonomy_archived(self): |
There was a problem hiding this comment.
| def test_patch_taxonomy_archived(self): | |
| def test_patch_archived_taxonomy_is_400(self): |
| response = self.client.delete(url) | ||
| assert response.status_code == status.HTTP_404_NOT_FOUND | ||
|
|
||
| def test_export_taxonomy_archived_404(self): |
There was a problem hiding this comment.
| def test_export_taxonomy_archived_404(self): | |
| def test_export_archived_taxonomy_responds_404(self): |
| object_tag_3.refresh_from_db() | ||
| self.assertEqual(object_tag_3.value, updated_tag_value) | ||
|
|
||
| def test_update_tag_in_taxonomy_archived_tag(self): |
There was a problem hiding this comment.
| def test_update_tag_in_taxonomy_archived_tag(self): | |
| def test_update_tag_in_taxonomy_archived_tag_is_400(self): |
| fungi.refresh_from_db() | ||
| assert fungi.value == "Fungi" | ||
|
|
||
| def test_create_tag_in_archived_taxonomy_404(self): |
There was a problem hiding this comment.
| def test_create_tag_in_archived_taxonomy_404(self): | |
| def test_create_tag_in_archived_taxonomy_is_400(self): |
| tag_count = self.small_taxonomy.tag_set.count() | ||
|
|
||
| response = self.client.post(self.small_taxonomy_url, {"tag": "New Tag"}, format="json") | ||
| assert response.status_code == status.HTTP_404_NOT_FOUND |
There was a problem hiding this comment.
I think 400 would be better here, because a 404 only shows that the tag doesn't exist, implying some kind of unclear error (tag created but not read? Create failed? Why?) - a 400 clearly responds that the tag creation is not allowed and that it did not go through.
There was a problem hiding this comment.
Agreed, and applied as one rule. Reads of an archived taxonomy return 404. Writes return 400 with an explanation: tag POST, PUT, and PATCH, taxonomy PUT and PATCH, and both import actions. Deletes keep their current behavior for openedx#780 to define, and tests pin it. (Claude generated comment)
| raise Http404("Taxonomy not found") | ||
| self.check_object_permissions(self.request, taxonomy) | ||
| if is_edit and taxonomy.archived: | ||
| raise ValidationError("This taxonomy is archived and cannot be edited.") |
There was a problem hiding this comment.
I'm thinking that get_object is a read method, while the ValidationError is supposed to be thrown when attempting to edit. Could we improve the separation of concerns here and throw this validation error in a method related to updating?
Something like this?
def perform_update(self, serializer) -> None:
if serializer.instance.archived:
raise ValidationError("This taxonomy is archived and cannot be edited.")
super().perform_update(serializer)
There was a problem hiding this comment.
Done. get_object only resolves the taxonomy; it resolves archived ones for the write and delete actions. perform_update refuses an archived taxonomy. The two import actions do the same check right after get_object, because they don't go through perform_update. (Claude Generated comment)
| taxonomy_id = self.kwargs.get("pk") | ||
| taxonomy = get_taxonomy(int(taxonomy_id)) if taxonomy_id else None | ||
| if taxonomy_id and self.request.method in ("PUT", "PATCH"): | ||
| taxonomy = Taxonomy.objects.filter(pk=int(taxonomy_id)).first() |
There was a problem hiding this comment.
Could we use semantic naming so that it's clear that this is about including versus excluding archived taxonomies?
taxonomy = get_taxonomy_including_archived(int(taxonomy_id))
``` for example?
There was a problem hiding this comment.
Done, as a private helper in views.py: _get_taxonomy_including_archived. I kept it out of api.py so the public API doesn't grow, and the documented opt-out for other code stays "use Taxonomy.objects". (Claude generated comment)
| """ | ||
| tag = self.tag_for_value(value) | ||
| if tag.archived or self.archived: | ||
| raise Tag.DoesNotExist(f"Tag '{value}' is archived or belongs to an archived taxonomy.") |
There was a problem hiding this comment.
Could we raise a more exact error, like Tag.IsArchived?
There was a problem hiding this comment.
Done. Tag.IsArchived subclasses Tag.DoesNotExist, so callers can tell archived apart from missing, while the existing except Tag.DoesNotExist handlers, the 404s on read paths, keep working. (Claude generated comment)
There was a problem hiding this comment.
Review with Fable
@tbain I did a code review with Fable that found quite a few things to go over that seem very worth it. Have a look, I think they all make a lot of sense. If you don't agree with something, feel free to push back.
I think if you pass these instructions here to AI verbatim (have Fable or Opus make a plan then Sonnet implement) it should be able to fix them quite autonomously.
Fable Recommendations:
Background: the models and terms the findings use (skip if you already know them)
The three models and the archived field. openedx_tagging has three models in src/openedx_tagging/models/base.py. A Taxonomy is a named set of tags, such as "Subject". A Tag is one value inside a taxonomy, such as "Biology", and a tag can have a parent tag, so tags form a tree. An ObjectTag is the link between one piece of content (identified by a string object_id) and one tag; the ticket and the PR call this link an "association". A free-text taxonomy has no Tag rows at all: its associations have tag=NULL. The stacked PR openedx#856 (ticket openedx#776) added a boolean archived field to all three models. Nothing in the product sets it to True yet. Tickets openedx#779 and openedx#780 will set it when a user tries to delete a record that a learner's mastery status still depends on: instead of deleting the record, they will archive it.
What your PR does. It hides archived rows on read paths and refuses edits to them. The mechanism is three Django QuerySet subclasses, one per model, each with an unarchived() method, installed as the model's default manager with as_manager(). So Tag.objects.all() still returns every row, archived or not, and Tag.objects.unarchived() hides archived tags and tags whose taxonomy is archived. Django builds a reverse-relation manager such as taxonomy.tag_set from the related model's default manager class, so taxonomy.tag_set.unarchived() works too.
The "opt-out", also called the "escape hatch". Ticket openedx#778 required that any filter you added have a documented way to bypass it, because the follow-up tickets need to find archived rows: openedx#779 to un-archive an association when it is re-applied, openedx#780 to delete archived tags and taxonomies, and openedx#781 to send an event for an archived taxonomy's associations. In your design the opt-out is "use the default manager without calling unarchived()", so ObjectTag.objects.filter(...) sees everything. The ticket also required that the opt-out be described in a comment on the GitHub issue openedx#778 itself, where the engineers picking up openedx#779 through openedx#781 will look.
Two fields on the taxonomy and tag APIs. tags_count is a field of the taxonomy API responses that counts the taxonomy's unarchived tags. A tag listing is normally one level deep; with ?full_depth_threshold=N, a result set of fewer than N tags comes back as the whole subtree in tree order ("full-depth" in the findings below), and ?parent_tag=<value> lists the children of one tag.
openedx-core versus edx-platform. This repository is a Django library. edx-platform, the Studio and LMS application, installs it and has its own app openedx/core/djangoapps/content_tagging, which mostly calls functions in openedx_tagging/api.py. In a few places it queries the three models directly, and those queries do not pick up your filter unless they call unarchived() themselves.
CBE. Competency-Based Education (CBE) is the learner mastery tracking in openedx_learning that the archived state exists to protect. A competency criterion (CompetencyCriterion) is a rule for when a learner has mastered a competency, and it points at an association through its object_tag field.
How the review tested your tests. For each line you changed, the reviewer broke it (for example, by deleting a .unarchived() call) and ran the test suite. If every test still passes with the line broken, no test protects that line. Findings 3 and 4 come from this method.
ADRs. An Architecture Decision Record is a short document in docs/openedx_tagging/decisions/ that records a design decision and the alternatives rejected. The repository's CLAUDE.md asks for a new ADR, in "Proposed" status, for impactful decisions.
Must fix before merge
1. The opt-out is not documented on the ticket, and the PR description says it is
Ticket openedx#778 required a comment on the issue describing how to reach archived rows (the opt-out), but its two comments ("closing the discussion period" and "Accepted.") do not. The PR description claims "The issue thread on openedx#778 has the same explanation", which is false. openedx#779, openedx#780 and openedx#781 each plan to use "the documented opt-out openedx#778 provides", so without the comment each may build its own.
What to do: post the opt-out comment on openedx#778 and remove the false sentence from the PR description.
What the comment should say
- The ticket required two things on the issue: name the escape hatch, and, if the existing code had no single place to hang the filter on, say so there.
Taxonomy.objects,Tag.objects,ObjectTag.objectsand the reverse managers return every row including archived ones..unarchived()is the filtered view.- The
api.pyfunctions have noinclude_archivedkeyword. - openedx#779's queryset
delete()override belongs onObjectTagQuerySet.
2. A new tag can still be created under an archived parent tag
Taxonomy.add_tag looks up the parent tag without the archived filter, so POST /taxonomies/<id>/tags/ with parent_tag_value set to an archived tag creates a visible child under a hidden parent. The PR description says "Normal use can't create that state", but this endpoint does. It also breaks the ticket's rule that nothing may "rename, re-parent, or otherwise modify a record that has been archived" (scenario "An archived record cannot be edited"). Two read paths then disagree about that child: the full-depth tag listing returns it, and the tag export skips it.
What to do: make add_tag reject an archived parent and add a test. Ask on openedx#778 what a listing should show for an unarchived child of an archived parent, then make the listing and the export agree.
Code details for the fix
- The lookup in
models/base.pyisself.tag_set.get(value__iexact=parent_tag_value), which uses the unfiltered manager. - Resolve the parent through
_unarchived_tag_for_value, the helper you already wrote. An archived parent then raisesTag.DoesNotExist, which the POST view already turns into a 404 "Parent Tag not found". - The full-depth listing returns the child with a
parent_valuenaming a tag that is not in the response. The export code underimport_export/skips the child. - Related and smaller: calling
add_tag_to_taxonomy()from Python on an archived taxonomy is not blocked either. Only the REST route is blocked, and only because the view's taxonomy lookup returns 404 first.
3. Several lines you changed have no test that fails when they break
With each of these breaks applied one at a time, all 327 tests in the three touched test modules still passed.
| Break that no test catches | Test to add |
|---|---|
Removing .unarchived() from Tag.child_count. |
One test of child_count and descendant_count with an archived child covers this break and the next. |
Removing .unarchived() from Tag.descendant_count. |
The same test. |
Removing .unarchived() from get_tags_count in rest_api/v1/serializers.py. This fallback is what the retrieve, PUT and PATCH responses use, because only the list view adds the tags_count annotation. Your list test passes even with the fallback broken. |
One test that retrieves a single taxonomy over the API and checks tags_count. |
Removing .unarchived() from the full-depth subtree query in _get_filtered_tags_deep, the branch used when ?parent_tag= is combined with full_depth_threshold. |
One full-depth parent_tag case with an archived descendant. |
Making TaxonomyView.get_object use the filtered lookup for destroy. The PR description says destroy "keeps its behavior" on an archived taxonomy, and the review confirmed that it returns 204 and deletes the row, but no test pins that. |
One test that DELETE on an archived taxonomy returns 204 and removes it. |
Making TaxonomyTagsView.get_taxonomy resolve archived taxonomies for DELETE too. DELETE on an archived taxonomy's tags currently returns 404 and leaves the tag, and nothing tests that either. |
One test that DELETE on the archived taxonomy's tags returns 404 and keeps the tag. |
4. The association filter has two conditions that no test tells apart, and the free-text case is untested
ObjectTagQuerySet.unarchived() hides an association when any of four conditions holds, but removing either of the last two (tag__taxonomy__archived, taxonomy__archived) on its own leaves every test passing. The free-text case, which only taxonomy__archived covers, works but is untested.
What to do: add one test that archives a free-text taxonomy that has an association, and asserts that get_object_tags and get_object_tag_counts leave the association out.
Code details for the fix
- The four conditions are the association's own
archived,tag__archived,tag__taxonomy__archived, andtaxonomy__archived. - Removing both of the last two failed one test; removing either alone failed none.
- A free-text association has no tag to follow, so
taxonomy__archivedis the only condition that hides it. The review probed this case by hand and it works.
5. edx-platform needs more than the one-line change the PR description names, and no companion issue exists
The PR description names one edx-platform follow-up, the tags_count annotation in TaxonomyOrgView. The review found two more places in openedx/core/djangoapps/content_tagging/api.py where archived data gets through. get_all_object_tags queries the models directly, so archived associations still reach library responses, exports and copy and paste, and a paste re-creates them as visible associations. set_exported_object_tags can create taxonomy-less association rows when the taxonomy is archived.
What to do: open an edx-platform issue covering all three and link it from the PR description.
Code details for the issue
get_all_object_tagsrunsObjectTag.objects.filter(...)directly, with a comment saying no core API function exists for it. It feeds library container responses, the content-tag export, and the OLX serialization used by copy and paste. On paste,tag_objectre-creates the archived associations as fresh, visible ones, which is the "reintroduces them on the next import" risk the ticket warns about. The fix is to call.unarchived()there.set_exported_object_tagscallsget_taxonomy_by_export_id, which now returnsNonefor an archived taxonomy, and then callstag_objectwithcreate_invalid=True, which creates association rows with no taxonomy. The review did not check whether anyone wants that behavior, so the issue should ask.- The issue should list the
tags_countfilter,.unarchived()inget_all_object_tags, and a decision onset_exported_object_tagsfor archived taxonomies.
Decisions to make or write down
6. Refused edits answer 400 on some endpoints and 404 on others
The current behavior on an archived taxonomy is:
| Request | Status | Why |
|---|---|---|
| PUT or PATCH on the taxonomy | 400 | The REST view enforces the archived rule for taxonomies. |
PUT on tags/import and tags/import/plan, which also change the taxonomy |
404 | These actions use the filtered lookup. |
| PUT or PATCH on the taxonomy's tags | 400 for a tag that exists, 404 for one that does not | tag_set.get runs before your archived check in update_tag. |
| POST or DELETE on the taxonomy's tags | 404 | TaxonomyTagsView.get_taxonomy does not return an archived taxonomy for these methods, so the view answers 404 first. |
The status depends on which code path rejects the request, not on one rule. The "cannot edit an archived taxonomy" rule exists only in the REST view, so the Django admin and Python code calling taxonomy.save() are not blocked, while the tag rule in the model method update_tag applies everywhere.
What to do: pick one rule, either every action on an archived taxonomy is 404 or every edit is 400, and record it, for example in the ADR from finding 9. The reviewer's pending PR comment asking for 400 on POST is the same decision.
Code details for the fix
- If you keep 400, add the two import actions (
tags/importandtags/import/plan) to the view'sis_editcheck. - The PR description says tag DELETE is "left to openedx#780", but openedx#780 does not say that.
7. Saving an object's tag list from the filtered read will delete the hidden associations
tag_object(object_id, taxonomy, tags) replaces an object's whole tag list for a taxonomy: it deletes every existing association that is missing from the new list. Callers build that list from get_object_tags, which this PR now filters, so the next ordinary save of an object deletes its hidden archived associations, and in the CBE app the deletion cascades to any CompetencyCriterion attached to them. There is no live impact yet because nothing sets archived=True.
What to do: record this on openedx#779, which has to define what a full-list replace does with an association it cannot see, and note in the PR description that openedx#779 must ship with or before openedx#780.
Code details
tag_objectreads the current associations with the unfiltered manager, so it sees the archived ones and deletes those absent from the new list.- Studio's tag drawer PUT and
create_competency_criteriononmainboth build the new list fromget_object_tags. - Before this PR,
get_object_tagsincluded the archived association, so a save kept it. CompetencyCriterion.object_tagcascades on delete.
8. The scenario "retrieving an archived tag directly" has no URL to test
No endpoint fetches a single tag, so the scenario cannot be tested as written.
What to do: say on openedx#778 that you read the scenario as ?parent_tag=<value> returning 404 for an archived tag.
Why that is the closest case
GET /taxonomies/<id>/tags/ always lists tags. The only way to ask for a specific tag is ?parent_tag=<value>, and your 404 there covers the scenario.
Improvements worth doing
9. Write a proposed ADR
The repository asks for an ADR for impactful decisions, and archiving is one: ADR 0006 says content tags linked to a deleted tag remain and are displayed, while archiving hides them, a state no ADR describes. The CBE app on main also handles archived references differently from tagging.
What to do: write the ADR in "Proposed" status. The reviewer's pending PR comment asking for a Tag.IsArchived exception is part of the same question.
What the ADR should record, and the CBE differences
- Archived as a state distinct from deleted, as an amendment to 0006.
- That a tag's or association's archived state is computed from its ancestors at query time instead of being copied onto it.
unarchived()plus unfiltered default managers, with a filtered default manager as the rejected alternative, and why it was rejected.- The 400-versus-404 rule from finding 6.
- The exception vocabulary CBE should reuse.
The CBE app on main filters its own archived field inline with archived=False and refuses archived references with a Django ValidationError, while tagging uses ValueError.
10. Skip the lineage cascade for a tag with no taxonomy
Tag.save cascades a change in a tag's lineage (its ancestry path) to its descendants. The replacement query filters on taxonomy=self.taxonomy, so for a tag with no taxonomy it quietly updates every taxonomy-less tag that shares the lineage prefix, across unrelated trees. Normal use never creates taxonomy-less tags, so this is hardening: skip the cascade when self.taxonomy_id is None.
Code details
- A tag's
lineagecolumn holds its ancestry path, such as "Root", tab, "Parent", tab, "ThisValue", tab. - The old line
self.taxonomy.tag_set.filter(...)crashed withAttributeErrorwhentaxonomywasNone. - The new line
Tag.objects.filter(taxonomy=self.taxonomy, lineage__startswith=...)runs withtaxonomy=Noneinstead of crashing.
Things the review looked at and left to other tickets
- Autocomplete now offers a tag whose association on this object is archived, and
tag_objectthen leaves the row archived, so the user sees no change. openedx#779 owns un-archiving on re-apply insidetag_object, which your PR does not touch. - The import actions
RenameTagandUpdateParentTag(inimport_export/actions.py) can still edit an archived tag. openedx#814 says import must not resolve an incoming row to an archived tag, and it can do that without undoing anything in your PR. Your description already discloses this. - Deleting an archived taxonomy returns 204 and hard-deletes it. openedx#780 owns that behavior. The missing test is in finding 3.
What passed
The branch merges cleanly onto upstream/main and onto the moved openedx#856 branch. makemigrations --check reports no changes for this repository's apps. lint-imports, pylint, pycodestyle, isort and mypy are clean. All 536 openedx_tagging tests and all 117 openedx_learning tests pass. Every acceptance scenario has at least one test that fails when its behavior is broken, with the exceptions listed in findings 3 and 4.
1e5403d to
c2dc21c
Compare
|
Claude generated comment for the updates pushed to address code review issues: Jesper's review summaryShortened the description from about 1,770 words to about 610 and ran Fable findings
|
c535db2 to
9ef1b4e
Compare
c2dc21c to
2bd9204
Compare
Description
Makes archived tagging records inert, so the
archivedflag from openedx#856 (openedx#776) actually hides something. An archived taxonomy, tag, or tag-to-content association (ObjectTag) stays in the database, so the learner mastery history that points at it stays intact. The tagging app no longer lists, searches, counts, exports, or edits it. Stacked on openedx#856; until openedx#856 merges, the diff also shows its commits.How records are hidden. Each model's queryset has an
unarchived()method that defines its rule in one place. A taxonomy is hidden when it is archived; a tag when it or its taxonomy is; an association when it, its tag, or its taxonomy is. Ancestors are checked at query time, so archiving a taxonomy does not touch its tags (as approved on openedx#655). The read functions inapi.py,Taxonomy.get_filtered_tags, the tag counts, and export callunarchived(). The default managers (Tag.objectsand the others) and reverse managers such astaxonomy.objecttag_setstill return every row; that is the opt-out openedx#779, openedx#780, and openedx#781 use.Why not a filtered default manager. Django builds reverse relations, uniqueness checks, and the admin on the default manager, so filtering it would hide archived rows from code that must see them: the lineage cascade in
Tag.save,tag_object(openedx#779), and import planning (openedx#814). Joins and annotations would still need explicit filters.What an archived record returns over REST. Reads report not found; writes are refused with an explanation:
?parent_tag=In Python,
Taxonomy.add_tagandupdate_tagraiseValueError, and read paths raiseTag.IsArchived, a subclass ofTag.DoesNotExist.Also:
?parent_tag=with an unknown value now returns 404 instead of 500.tag_objectis unchanged because openedx#779 owns its archive behavior; and because a full-list replace through it deletes archived associations the caller can no longer see, openedx#779 must ship with or before openedx#780.Supporting information
Testing instructions
pytest tests/openedx_tagging tests/openedx_learning -o addopts=""(666 tests), covering each acceptance scenario plus query-count assertions onget_object_tag_countsandadd_usage_counts.python manage.py makemigrations oel_tagging --check --dry-runreports no changes.Tag.objects.filter(value="X").update(archived=True)), then check the table above against Studio's/api/content_tagging/v1/. I ran this as a Postman collection in tutor dev; all passed except thetags_countgap below.Deadline
None.
Other information
TaxonomyOrgViewcountstags_countwith its own annotation, so Studio's count still includes archived tags;get_all_object_tagsqueriesObjectTagwithoutunarchived(), andset_exported_object_tagsneeds a decision for archived taxonomies.parent_value, while export skips it. Only direct database edits create that state today.🤖 Generated with Claude Code