Skip to content

A foreign Skill/presentation + Skill/slide row at Version=1 beats the Skill package's install (split from #2361) #2403

Closed

Description

@rbuergi

Split out of #2361, whose installer-side defect is fixed by #2402. This is the other half: something writes a Skill/presentation (and Skill/slide) row at Version=1, carrying what looks like Publish/Skill/presentation's instructions and no Category/Order, while the Skill package is installing.

The evidence

Two MeshWeaver.Plugins gate runs on 2026-08-26 (job 98168018140 on run 32965818521, and publish-bake job 98177992724 on run 32968600070) carry this inside the Skill package's FIRST install, ~330 ms after ── Skill: installing 23 file(s)…:

[MonotonicWriteGuard] CONFLICT on Skill/presentation: a write at Version=0 lost the race against the durable Version=1.
  Resolved by merging into the durable row: merged=(none); latest-wins (stale values DROPPED)=Category, Order, Content.instructions.
[MonotonicWriteGuard] CONFLICT on Skill/slide: … (identical)

The write at Version=0 is the installer's bulk write of the package's own node. So a durable row at Version=1 already existed, and it is not the package's node.

MeshNodeConflictMerge reports exactly the members that diverged, which pins the shape of that row:

  • Name, Description, Icon, NodeType are NOT reported ⇒ they match the package's node (/presentation, Presentation, Skill).
  • Category and Order ARE reported ⇒ they differ. All three authored copies in the repo (Skill/, Publish/Skill/, Essentials/Skill/) carry identical category: Skills and order: 17/16, so the durable row carries neither — it looks like they were LOST somewhere, not set to something else.
  • Content.instructions IS reported, i.e. the two strings are not resolvable by MeshNodePatchMerge.TryMergeTwoWay's single-splice superset rule. Of the three copies only Publish's has that property against Skill's: Essentials/Skill/presentation's instructions are a clean single-splice subset of Skill/presentation's (they would have MERGED and been reported under merged=), while Publish's diverges in the middle as well (PublishAndDecks vs SlidesAndDecks).

Publish installs immediately before Skill in the gate's dependency order, and presentation + slide are EXACTLY the two node ids Publish/Skill/ and Skill/ have in common — which is also exactly the pair the gate reports, every time.

What it is NOT

  • Not two packages declaring one path. PackageInstaller.ParseCanonical derives the node path from the file's repo-relative location, so Publish/Skill/presentation.json → Publish/Skill/presentation.
  • Not BuiltInSkillProvider. The tester boots .AddAI(AiContentSources.ContentPartitions), so AddSkillType takes its dbSynced branch and registers no static node provider for Skill; its IConfiguration is empty, so no SkillStaticRepoSource import is registered either.

Not reproduced locally

Three full 53-package gate runs against the plugins checkout (one at DOTNET_PROCESSOR_COUNT=4), each with a probe printing every InMemoryStorageAdapter.Write for any path containing Skill/presentation, saw exactly ONE write per path — the install's own. The extra writer did not appear on this host.

Suggested next diagnostic

The cheapest next step is to make the conflict say WHAT the durable row is: MonotonicWriteGuardStorageAdapter.ResolveConflict already has latest in hand, so logging its Name/Category/NodeType (and a short content fingerprint) alongside the member list would identify the writer from the next CI occurrence without a repro. The guard also writes an ActivityLog satellite at {path}/_Activity/write-conflict-{version} — that node already carries the account and is another place to look on a portal where this happens.

Activity

  1. rbuergi commented on Aug 26, 2026

    @rbuergi
    ContributorAuthor

    The writer is identified: Store/Publishing's pre-installed-skill MIRROR, in MeshWeaver.Plugins

    Skill/presentation and Skill/slide at Version=1 are written by
    Store/Publishing/Source/PreInstall.cs
    — in-mesh C# in the MeshWeaver.Plugins node repo, so it is invisible to dotnet build, to any
    grep --include='*.cs' over this repo, and to every hypothesis this issue and #2361 tested.

    public const string GlobalSkillNamespace = "Skill";
    public static string MirrorPath(string skillId) => $"{GlobalSkillNamespace}/{skillId}";
    
    public static MeshNode Mirror(MeshNode skill) =>
        new(skill.Id, GlobalSkillNamespace)          // ← namespace REBASED from "Publish/Skill" to "Skill"
        {
            Name = skill.Name,
            Description = skill.Description,
            NodeType = skill.NodeType,
            Icon = skill.Icon,
            MainNode = skill.Path,                   // ← provenance: "Publish/Skill/presentation"
            Content = skill.Content,
        };                                            // ← Category and Order are NOT copied

    A plugin declaring preInstalled: true has its shipped {plugin}/Skill/* mirrored into the global
    Skill/ registry (the namespace the composer's canonical skill query reads for every user).
    PluginGating.cs drives it from the plugin root hub — DelaySubscription + Throttle(2s) — and
    SyncGlobalSkills writes the plan through mesh.CreateNode under system-security.

    That accounts for every attribute the guard reported, with nothing left over:

    Observation Explained by
    Version=1 CreateNode on a brand-new path mints MeshNode.NextVersion(0) = 1
    Category diverged Mirror() does not copy Category — the mirror's is null, the package's is "Skills"
    Order diverged same — the mirror's Order is null, the package's 17 / 16
    Name / Description / Icon / NodeType NOT reported Mirror() copies all four verbatim
    Content.instructions unmergeable it is Publish's body, which diverges from the master's mid-string
    only presentation + slide the only two ids Publish/Skill/ and the Skill package have in common
    intermittent the mirror is delayed + throttled off the plugin hub's own subtree stream, so whether it lands before the Skill package's install is a race with the gate's sequential install progress

    The proof, not the inference

    1. The durable content is byte-for-byte Publish's, not Essentials' and not core's
    content/ai/Skill/*.md. Plugins CI run
    32996370339, with
    MW_INSTALL_DIFF=1 armed by MeshWeaver.Plugins@3ddf7727, prints the canonical content signature
    lengths:

    [DIFF] Skill/presentation scalars=False lens=5016/7138 firstDiff@60
    [DIFF] Skill/slide        scalars=False lens=8178/10300 firstDiff@60
    

    Recomputing ContentSignature over every candidate file (canonical JSON, .NET default escaping)
    resolves it uniquely — the two-character PublishAndDecks vs SlidesAndDecks difference is exactly
    what separates the winner from the runners-up:

    candidate presentation slide
    Skill/*.json (the master — the incoming side, 7138 / 10300) 7138 10300
    Publish/Skill/*.json 5016 ✅ 8178 ✅
    Essentials/Skill/*.json 5014 8176
    core content/ai/Skill/*.md (pre-preamble) 5014 8176

    2. The mirror was caught in the act, locally, with a probe on InMemoryStorageAdapter.Write
    (path + scalars + stack) and on HandleCreateNodeRequest (sender). A full 53-package gate run
    against the plugins checkout shows the mirrors as a distinct population — v1, cat='', order=,
    ns='Skill' — beside the packages' own v0 rows:

    Publish/Skill/multipart-doc  v0 ns='Publish/Skill' cat=''     order=  name='/multipart-doc'
    Skill/multipart-doc          v1 ns='Skill'         cat=''     order=  name='/multipart-doc'   ← mirror
    Skill/presentation           v0 ns='Skill'         cat='Skills' order=17 name='/presentation' ← the package
    Skill/thread                 v1 ns='Skill'         cat=''     order=  name='/thread'          ← mirror (Collaboration)
    Skill/ui-extensibility       v1 ns='Skill'         cat=''     order=  name='/ui-extensibility'← mirror (Hosting)
    

    and the create-side probe names the poster:

    [CREATETRACE] Skill/thread            sender=Collaboration  caller=system-security
    [CREATETRACE] Skill/ModelForeignKeys  sender=DataModelling  caller=system-security
    [CREATETRACE] Skill/create-course     sender=Edu            caller=system-security
    

    Every mirrored id belongs to a preInstalled: true plugin (Collaboration, DataModelling, Edu,
    Feedback, Hosting, Observability, Publish); no id from a non-pre-installed one (Providers,
    RemoteControl, RolePlay, Store, Training, Video) is mirrored. The bursts land tens of seconds after
    their own package's install — which is the DelaySubscription + Throttle — and that lateness is
    the race.

    Three local runs stayed GREEN for the same reason the CI runs went red: locally the Publish mirror
    fired after the Skill package had installed, so MirrorPlan saw a FOREIGN occupant at
    Skill/presentation and correctly left it alone (IsOwnMirror is false — its MainNode is its own
    path). In CI it fired first, created the row, and the package's Version=0 bulk write then lost to
    it exactly as #2361 described.

    What this was NOT (each ruled out with evidence, so nobody re-tests them)

    • Not BuiltInSkillProvider / a static node provider. The gate boots
      .AddAI(AiContentSources.ContentPartitions), so AddSkillType takes its dbSynced branch and
      registers neither the IStaticNodeProvider nor the StaticNodePartitionStorageProvider; the
      probe run shows no static write at those paths. And the pinned image (b2e1ff9, ACR
      3.0.0-rc8.ci.5285) contains Skill catalog: port the write-into-your-own-home preamble to the canonical /presentation + /slide seeds #2187, so its content/ai/Skill/presentation.md body is byte-equal
      to the master package's — it could not have produced a divergent Content.instructions at all.
    • Not a path-derivation bug. NodeFileMapper.FromRelativePath runs on the REPO-relative path
      (NodeRepoPackageSource.FetchPackageFiles keeps the Publish/ prefix), so
      Publish/Skill/presentation.json → Publish/Skill/presentation, confirmed by the trace above.
    • Not the installer. Plugin install: a bulk write the store REFUSED is no longer reported as written (#2361) #2402 fixed the installer half; the losing write is its bulk write at the
      version the repo file carries, which is correct.

    Status and follow-ups

    • The collision is already gone: MeshWeaver.Plugins#714 (merged 2026-08-26 19:35Z) deleted the
      21 duplicate skill copies — Publish/Skill/presentation.json and slide.json among them — and
      added check_one_skill_master to scripts/validate-repos.py, which refuses any package other
      than Skill/ re-shipping a master id. That is the right resolution: one master per platform
      skill, gated.
    • Core: the guard's warning and its durable _Activity/write-conflict-{n} record now carry the
      winning row's MainNode. A mainNode pointing outside the path's own partition names a
      cross-partition mirror on sight — which is the one line that would have ended this hunt
      immediately. (LastModifiedBy cannot: every system-impersonated writer stamps system-security.)
    • MeshWeaver.Plugins, separate defect found on the way: PreInstall.Mirror drops Category and
      Order, so every mirrored skill currently sits in the global registry with no category and
      order 0 — visible in the trace above for Skill/multipart-doc, Skill/thread,
      Skill/ui-extensibility and the rest. MirrorEquals does not compare them either, so the loss is
      not self-healing. Worth fixing there; it does not affect this issue's outcome.
  2. rbuergi commented on Aug 26, 2026

    @rbuergi
    ContributorAuthor

    Follow-ups now tracked:

    • Core — A write conflict names the writer: the durable row's provenance, in the warning and the trace #2440 puts the durable row's MainNode in the conflict warning and in the _Activity/write-conflict-{n} trace, so a cross-partition mirror is named on sight. Pinned by a test that fails on main.
    • MeshWeaver.Plugins — Systemorph/MeshWeaver.Plugins#721 for the lossy PreInstall.Mirror (drops Category + Order, and MirrorEquals does not compare them, so it never heals).
    • The collision itself is already closed by Systemorph/MeshWeaver.Plugins#714 (one master per platform skill, gated by check_one_skill_master).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions