Set Verified Only When a Verification Ran

ARCHITECTURE.md

@@ -91,7 +91,7 @@ Critical performance feature - DO NOT break compatibility:
 - Contains: processing state, tool versions, media properties, file hash
 - Hash: First 64KB + last 64KB of file (not timestamp-based)
 - Schema versioned (`SchemaVersion: 5` in `SidecarFileJsonSchema5`, global alias in `GlobalUsings.cs`)
-- A `Verified` or `VerifyFailed` state skips only the checks that read it, see [Sidecar Refresh](#sidecar-refresh)
+- A `Verified`, `VerifyFailed` or scanned state skips only the checks that read it, see [Sidecar Refresh](#sidecar-refresh)
 - State flags are bitwise: `StatesType` enum with `[Flags]` attribute
 - Sidecar operations: `Open()`, `Create()`, `Read()`, `Update()`, see [Sidecar Refresh](#sidecar-refresh)
 
@@ -372,7 +372,7 @@ flowchart TD
 
 ### Sidecar Refresh
 
-Every step that changes the file calls `Refresh()`. `Refresh(newName)` adds `FileReNamed` when the path changed, then reloads the media info. `GetMediaProps()` calls `Refresh(false)` and then fails a file that any tool reports as unsupported.
+Every step that changes the file calls `Refresh()`. `Refresh(newName)` adds `FileReNamed` when the path changed, then reloads the media info. `GetMediaProps()` calls `Refresh(false)` and then fails a file that any tool reports as unsupported. A step that only changes the state calls `SaveState()`, which calls `Refresh(false)` only with `UseSidecarFiles` on.
 
 ```mermaid
 flowchart TD
@@ -390,7 +390,7 @@ flowchart TD
   M1 -->|no| M2["State becomes FileModified, all other flags reset"]
   M1 -->|yes| T1
   M2 --> T1{"SidecarUpdateOnToolChange on and IsToolsCurrent() false?"}
-  T1 -->|yes| T2["Clear Verified"]
+  T1 -->|yes| T2["Clear Verified, ScannedInterlace and ScannedCaptions"]
   T1 -->|no| T3
   T2 --> T3{"Media changed, or the tool change above applied?"}
   T3 -->|yes| T4["Treat as modified"]
@@ -402,7 +402,7 @@ flowchart TD
   classDef suspect stroke:#c00,stroke-width:2px,stroke-dasharray:5 3
 ```
 
-`IsMediaCurrent()` logs a changed write time and ignores it, because the write time is unreliable over SMB. The sidecar holds media info and state flags, and it never skips a whole file. A `Verified` or `VerifyFailed` flag skips only the checks that read it, such as verify and the scans for interlacing and closed captions.
+`IsMediaCurrent()` logs a changed write time and ignores it, because the write time is unreliable over SMB. The sidecar holds media info and state flags, and it never skips a whole file. A `Verified` or `VerifyFailed` flag skips only the checks that read it, such as verify and the scans for interlacing and closed captions. A clean scan of the whole file sets `ScannedInterlace` or `ScannedCaptions`, which skips only that scan. A later step that rewrites the file keeps both flags, because no step adds captions or interlacing to the media it writes.
 
 ### Remux and Container Steps
 
@@ -462,7 +462,7 @@ flowchart TD
 
 ### Metadata and Structure Repair
 
-`RepairMetadataErrors()` repairs the errors the media tools flagged while parsing. `SetVerifyFailed()` sets `VerifyFailed`, clears `Verified` and fails the file, which stops later runs from retrying the repair. With the `Verify` option off, the next run clears `VerifyFailed` again, S4.
+`RepairMetadataErrors()` repairs the errors the media tools flagged while parsing. `SetVerifyFailed()` sets `VerifyFailed`, clears `Verified` and fails the file, which stops later runs from retrying the repair.
 
 ```mermaid
 flowchart TD
@@ -520,11 +520,12 @@ flowchart TD
   A1 -->|yes| F0["FindClosedCaptionTracks()"]
   F0 --> F1{"Any tool flags captions on a video track?"}
   F1 -->|yes| FOUND
-  F1 -->|no| F2{"State has Verified, VerifyFailed or ClearedCaptions?"}
-  F2 -->|"yes, S4"| AOK
+  F1 -->|no| F2{"State has ScannedCaptions, Verified, VerifyFailed or ClearedCaptions?"}
+  F2 -->|yes| AOK
   F2 -->|no| F3["FfProbe caption scan"]
   F3 -->|scan fails| FAIL(["Fail"])
-  F3 -->|none| AOK
+  F3 -->|none| F4["Without --quickscan, sets ScannedCaptions and saves state"]
+  F4 --> AOK
   F3 -->|found| FOUND["Captions found on the first video track"]
   FOUND --> C1{"FfMpeg has a NAL unit for this video format?"}
   C1 -->|no| FAIL
@@ -550,11 +551,12 @@ flowchart TD
   F1 -->|yes| FOUND
   F1 -->|no| F2{"--quickscan set?"}
   F2 -->|yes| AOK
-  F2 -->|no| F3{"State has Verified, VerifyFailed or DeInterlaced?"}
-  F3 -->|"yes, S4"| AOK
+  F2 -->|no| F3{"State has ScannedInterlace, Verified, VerifyFailed or DeInterlaced?"}
+  F3 -->|yes| AOK
   F3 -->|no| F4["FfMpeg idet scan"]
   F4 -->|"scan fails or no result"| FAIL(["Fail"])
-  F4 -->|progressive| AOK
+  F4 -->|progressive| F5["Sets ScannedInterlace and saves state"]
+  F5 --> AOK
   F4 -->|interlaced| FOUND["Interlaced, warns when DeInterlaced is already set, S9"]:::suspect
   FOUND --> D1["HandBrake deinterlaces to .tmp8, subtitles excluded"]
   D1 -->|fail| FAIL
@@ -648,9 +650,9 @@ The `ConvertOptions` settings shape the FfMpeg and HandBrake command lines, and
 
 ```mermaid
 flowchart TD
-  V0["Verify()"] --> V1{"Verify option on?"}
-  V1 -->|"no, S4"| PASS
-  V1 -->|yes| V2{"State has Verified?"}
+  V0["VerifyAndRepair()"] --> V1{"Verify option on?"}
+  V1 -->|"no, state unchanged"| OK
+  V1 -->|yes| V2{"Verify(): state has Verified?"}
   V2 -->|yes| PASS["Verify passes"]
   V2 -->|no| V3{"State has VerifyFailed?"}
   V3 -->|yes| NOREP["Verify fails, canRepair false"]
@@ -664,7 +666,7 @@ flowchart TD
   V7 -->|"ToolError, FfMpeg did not launch, was cancelled, or threw"| TE["Verify fails, the media unchecked"]
   PASS --> P1{"Verified already set?"}
   P1 -->|yes| OK(["Continue"])
-  P1 -->|no| P2["Set Verified, clear VerifyFailed, and PostDecodeTimestamps with the Verify option on, save state"]
+  P1 -->|no| P2["Set Verified, clear VerifyFailed and PostDecodeTimestamps, save state"]
   P2 --> OK
   REP --> C0{"Cancelled?"}
   C0 -->|yes| FAIL(["Fail, state unchanged"])
@@ -707,7 +709,7 @@ flowchart TD
   classDef suspect stroke:#c00,stroke-width:2px,stroke-dasharray:5 3
 ```
 
-Every repair failure sets `RepairFailed` and `VerifyFailed` together, so the `RepairFailed` check before a repair usually changes nothing. The exception is S4, where a pass with the `Verify` option off sets `Verified`, clears `VerifyFailed` and keeps `RepairFailed` and `PostDecodeTimestamps`. A later tool change with `SidecarUpdateOnToolChange` on clears `Verified`, and the next failed verify then skips its repair. With the `Verify` option on, a file with `VerifyFailed` fails again on each later run until its media changes and the sidecar resets to `FileModified`. `RegisterInvalidFiles` adds such a file to `FileIgnoreList` instead, so later runs ignore it.
+Every repair failure sets `RepairFailed` and `VerifyFailed` together, and `VerifyFailed` fails the verify before any repair. So the `RepairFailed` check before a repair matters only for a sidecar an earlier version wrote, which can hold `RepairFailed` without `VerifyFailed`. With the `Verify` option off, `VerifyAndRepair()` leaves the state unchanged, so `Verified` is set only by a verification that ran. With the `Verify` option on, a file with `VerifyFailed` fails again on each later run until its media changes and the sidecar resets to `FileModified`. `RegisterInvalidFiles` adds such a file to `FileIgnoreList` instead, so later runs ignore it.
 
 ### State Flags by Step
 
@@ -722,21 +724,20 @@ The table lists the `StatesType` flags each step sets, and the flags it clears.
 | `RemoveTags()` | `RemovedAttachments`, `ClearedTags` | |
 | `RepairMetadataErrors()` | `SetLanguage`, `ReMuxed`, `SetFlags`, `VerifyFailed` | `Verified` |
 | `RepairMatroskaStructure()` | `ReMuxed`, `VerifyFailed` | `Verified` |
-| `RemoveClosedCaptions()` | `ClearedCaptions`, `ReMuxed` | |
-| `DeInterlace()` | `DeInterlaced`, `ReMuxed` | |
+| `RemoveClosedCaptions()` | `ScannedCaptions`, `ClearedCaptions`, `ReMuxed` | |
+| `DeInterlace()` | `ScannedInterlace`, `DeInterlaced`, `ReMuxed` | |
 | `SetUnknownLanguageTracks()` | `SetLanguage` | |
 | `RemoveUnwantedLanguageTracks()`, `RemoveDuplicateTracks()` | `ReMuxed` | |
 | `RepairDefaultFlags()` | `ClearedDefaultFlags` | |
 | `ReEncode()` | `ReEncoded`, `ReMuxed` | |
 | `VerifyAndRepair()` | `Verified`, `VerifyFailed`, `Repaired`, `RepairFailed`, `PostDecodeTimestamps`, `BitrateExceeded` | `Verified`, `VerifyFailed`, `Repaired`, `RepairFailed`, `PostDecodeTimestamps` |
-| `SidecarFile.Read()` | `FileModified` when the media changed | every other flag when the media changed, `Verified` on a tool change with `SidecarUpdateOnToolChange` on |
+| `SidecarFile.Read()` | `FileModified` when the media changed | every other flag when the media changed, `Verified`, `ScannedInterlace` and `ScannedCaptions` on a tool change with `SidecarUpdateOnToolChange` on |
 
 ### Suspect Branches
 
 Each entry describes a branch the diagrams mark, as the code behaves today, and links the issue that tracks its fix. Update the diagram and remove the entry when the fix lands.
 
 - **S3** ([#1070](https://github.com/ptr727/PlexCleaner/issues/1070)): Deleting an orphan sidecar returns false, so the file counts as an error and runs `DeleteFailedFile()`. Deleting a non-MKV file in `DeleteNonMkvFile()` counts as a success.
-- **S4** ([#1071](https://github.com/ptr727/PlexCleaner/issues/1071)): With the `Verify` option off, `Verify()` passes and `VerifyAndRepair()` still sets `Verified`. Enabling `Verify` later skips those files, and `Verified` also skips the interlace and caption scans. A metadata repair that sets `VerifyFailed` is retried on every other run, since the next pass clears it.
 - **S6** ([#1073](https://github.com/ptr727/PlexCleaner/issues/1073)): `Convert.ReMuxToMkv()` moves its output to the `.mkv` name with overwrite, so `name.avi` replaces an existing `name.mkv`. `MakeExtensionLowercase()` replaces a lowercase sibling on a case-sensitive filesystem the same way.
 - **S7** ([#1074](https://github.com/ptr727/PlexCleaner/issues/1074)): `RepairMetadataErrors()` can pass with errors left. `RepairMetadataRemux()` does nothing with `ReMux` off, and `RepairMetadataFlags()` neither re-checks its result nor marks `VerifyFailed`.
 - **S8** ([#1075](https://github.com/ptr727/PlexCleaner/issues/1075)): The cleanup re-run after `VerifyAndRepair()` repeats only metadata, language, tag and default flag cleanup. It skips language and duplicate track removal, captions, cover art and the structure check after a re-encode repair.
@@ -833,7 +834,7 @@ enum StatesType {
     BitrateExceeded, ClearedTags, FileReNamed, FileDeleted,
     FileModified, ClearedCaptions, RemovedAttachments,
     SetFlags, RemovedCoverArt, ClearedDefaultFlags,
-    PostDecodeTimestamps
+    PostDecodeTimestamps, ScannedInterlace, ScannedCaptions
 }
 ```
 
@@ -964,7 +965,7 @@ Two entry-point workflows, whose build and publish jobs are thin callers of the
 
 ### Performance Considerations
 
-- Sidecar files enable fast re-processing (cached media info, and a `Verified` state skips verification and the scans)
+- Sidecar files enable fast re-processing (cached media info, and a `Verified` or scanned state skips verification and the scans)
 - `--parallel` most effective with I/O-bound operations (re-mux)
 - `--quickscan` limits scan to 3 minutes (trades accuracy for speed)
 - `--testsnippets` creates 30s clips for testing

HISTORY.md

@@ -59,6 +59,12 @@ Utility to optimize media files for Direct Play in Plex, Emby, Jellyfin, etc.
     - The next run retries the repair.
   - A re-encode or deinterlace replaces the media file only after every rewriting step succeeds, the track language restore included.
     - Before, a failure in a later step left the rewritten file with its state unchanged and reported it as not modified.
+  - With the `Verify` option off, processing leaves a file's state unchanged rather than marking it `Verified`.
+    - Enabling `Verify` later verifies such a file, and a file marked `VerifyFailed` keeps that state.
+    - The `ScannedInterlace` and `ScannedCaptions` state flags record a clean interlace or closed caption scan of the whole file.
+    - So each scan runs once per file without a verify, and a `--quickscan` sample sets neither flag.
+    - A file an earlier version marked `Verified` with the `Verify` option off is not re-verified automatically, and the `createsidecar` command resets its state.
+  - With `UseSidecarFiles` off, a step that only changes a file's state does not run the media info tools again.
 - Version 3.22:
   - Added always-on runtime metrics published via `System.Diagnostics.Metrics` under the `PlexCleaner.Process` meter, readable with `dotnet-counters` with no extra infrastructure.
     - Overall progress is operation-weighted: each heavy full-file operation (the closed-caption and interlace scans, bitrate analysis, re-encode, deinterlace, and verify) counts the file's size as work to do when it starts and as work done when it finishes, so a run mixing tiny and huge files reports the actual work completed, and the total grows as the non-deterministic per-file path is discovered.

PlexCleaner/ProcessFile.cs

@@ -405,7 +405,7 @@ private bool SetVerifyFailed(string operation)
         );
         _sidecarFile.State |= SidecarFile.StatesType.VerifyFailed;
         _sidecarFile.State &= ~SidecarFile.StatesType.Verified;
-        _ = Refresh(false);
+        _ = SaveState();
 
         // Always returns false to signal the repair did not succeed
         return false;
@@ -1168,11 +1168,11 @@ out FfMpegIdetInfo? idetInfo
             return true;
         }
 
-        // Running idet is expensive, skip if already verified or already deinterlaced
         if (
             conditional
             && (
-                State.HasFlag(SidecarFile.StatesType.Verified)
+                State.HasFlag(SidecarFile.StatesType.ScannedInterlace)
+                || State.HasFlag(SidecarFile.StatesType.Verified)
                 || State.HasFlag(SidecarFile.StatesType.VerifyFailed)
                 || State.HasFlag(SidecarFile.StatesType.DeInterlaced)
             )
@@ -1194,7 +1194,7 @@ out FfMpegIdetInfo? idetInfo
         {
             // Not interlaced, clear the idet info so the caller does not treat it as a detection
             idetInfo = null;
-            return true;
+            return SetScanned(SidecarFile.StatesType.ScannedInterlace);
         }
 
         // Use the first video track from FfProbe
@@ -1226,11 +1226,11 @@ private bool FindClosedCaptionTracks(bool conditional, out VideoProps? videoProp
             return true;
         }
 
-        // Running analyze_frames is expensive, skip if already verified or closed captions already removed
         if (
             conditional
             && (
-                State.HasFlag(SidecarFile.StatesType.Verified)
+                State.HasFlag(SidecarFile.StatesType.ScannedCaptions)
+                || State.HasFlag(SidecarFile.StatesType.Verified)
                 || State.HasFlag(SidecarFile.StatesType.VerifyFailed)
                 || State.HasFlag(SidecarFile.StatesType.ClearedCaptions)
             )
@@ -1258,9 +1258,16 @@ private bool FindClosedCaptionTracks(bool conditional, out VideoProps? videoProp
             // Use the first video track from FfProbe
             videoProps = FfProbeProps.Video.First();
             videoProps.ClosedCaptions = true;
+            return true;
         }
 
-        return true;
+        return Program.Options.QuickScan || SetScanned(SidecarFile.StatesType.ScannedCaptions);
+    }
+
+    private bool SetScanned(SidecarFile.StatesType scanned)
+    {
+        _sidecarFile.State |= scanned;
+        return SaveState();
     }
 
     public bool DeInterlace(bool conditional, ref bool modified)
@@ -1703,13 +1710,6 @@ public bool Verify(bool conditional, out bool canRepair)
         canRepair = false;
         _lastVerifyResult = VerifyResult.Clean;
 
-        // Conditional
-        if (conditional && !Program.Config.ProcessOptions.Verify)
-        {
-            // Done
-            return true;
-        }
-
         // Skip if Verified
         if (conditional && _sidecarFile.State.HasFlag(SidecarFile.StatesType.Verified))
         {
@@ -1821,6 +1821,11 @@ public bool DeleteFailedFile()
 
     public bool VerifyAndRepair(ref bool modified)
     {
+        if (!Program.Config.ProcessOptions.Verify)
+        {
+            return true;
+        }
+
         // Verify (verify does not set any state, it just tests state)
         if (Verify(true, out bool canRepair))
         {
@@ -1829,14 +1834,11 @@ public bool VerifyAndRepair(ref bool modified)
             {
                 _sidecarFile.State |= SidecarFile.StatesType.Verified;
                 _sidecarFile.State &= ~SidecarFile.StatesType.VerifyFailed;
-                if (Program.Config.ProcessOptions.Verify)
-                {
-                    _sidecarFile.State &= ~SidecarFile.StatesType.PostDecodeTimestamps;
-                }
+                _sidecarFile.State &= ~SidecarFile.StatesType.PostDecodeTimestamps;
                 Debug.Assert(!_sidecarFile.State.HasFlag(SidecarFile.StatesType.RepairFailed));
 
                 // Update state
-                return Refresh(false);
+                return SaveState();
             }
 
             // Done
@@ -1872,7 +1874,7 @@ public bool VerifyAndRepair(ref bool modified)
                 Debug.Assert(!_sidecarFile.State.HasFlag(SidecarFile.StatesType.Repaired));
 
                 // Update state
-                _ = Refresh(false);
+                _ = SaveState();
             }
 
             // Error
@@ -1915,7 +1917,7 @@ public bool VerifyAndRepair(ref bool modified)
             _sidecarFile.State &= ~SidecarFile.StatesType.Repaired;
 
             // Update state
-            _ = Refresh(false);
+            _ = SaveState();
 
             // Error
             return false;
@@ -2255,7 +2257,7 @@ public bool RepairTimestamps(ref bool modified)
                 _sidecarFile.State &= ~SidecarFile.StatesType.VerifyFailed;
                 _sidecarFile.State &= ~SidecarFile.StatesType.RepairFailed;
                 _sidecarFile.State &= ~SidecarFile.StatesType.PostDecodeTimestamps;
-                return Refresh(false);
+                return SaveState();
             case VerifyResult.TimestampOnly:
                 // Detected non-monotonic DTS, run the timestamp repair ladder
                 return RepairTimestampsAndSetState(ref modified);
@@ -2311,7 +2313,7 @@ private bool RepairTimestampsAndSetState(ref bool modified)
         {
             _sidecarFile.State |= SidecarFile.StatesType.PostDecodeTimestamps;
         }
-        _ = Refresh(false);
+        _ = SaveState();
         return false;
     }
 
@@ -2439,6 +2441,9 @@ private bool Refresh(string fileName)
         return Refresh(true);
     }
 
+    // Without a sidecar the state is not persisted, and a Refresh() would probe the unchanged media again
+    private bool SaveState() => !Program.Config.ProcessOptions.UseSidecarFiles || Refresh(false);
+
     private bool Refresh(bool modified)
     {
         // Call Refresh() at each processing function exit

PlexCleaner/SidecarFile.cs

@@ -31,6 +31,8 @@ public enum StatesType
         RemovedCoverArt = 1 << 16,
         ClearedDefaultFlags = 1 << 17,
         PostDecodeTimestamps = 1 << 18,
+        ScannedInterlace = 1 << 19,
+        ScannedCaptions = 1 << 20,
     }
 
     private const string SidecarExtension = ".PlexCleaner";
@@ -151,15 +153,16 @@ public bool Read(out bool current, bool verify = true)
             && Program.Config.ProcessOptions.SidecarUpdateOnToolChange
         )
         {
-            // Remove the verified state flag if set
             current = false;
-            if (State.HasFlag(StatesType.Verified))
+            const StatesType toolStates =
+                StatesType.Verified | StatesType.ScannedInterlace | StatesType.ScannedCaptions;
+            if ((State & toolStates) != 0)
             {
                 Log.Warning(
-                    "Sidecar out of sync with tools, clearing Verified flag : {FileName}",
+                    "Sidecar out of sync with tools, clearing Verified and Scanned flags : {FileName}",
                     _sidecarFileInfo.FullName
                 );
-                State &= ~StatesType.Verified;
+                State &= ~toolStates;
             }
         }