Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -428,7 +428,7 @@ flowchart TD

The remux steps rewrite the container and set `ReMuxed`. `RemoveTags()` and `RemoveAttachments()` edit the file in place with MkvPropEdit.

Every step that rewrites the file writes its output through MkvMerge, the lossless timestamp repair excepted. That MkvMerge pass uses `MkvMergeBuilder.InputOptions.Default()`, which drops global tags, track tags, attachments, and buttons. That covers remux, track removal, re-encode, deinterlace, closed caption removal, repair by re-encode, and the cover art remux, the FfMpeg remux fallback included. The pass drops them whatever `RemoveTags` says, so an FfMpeg or HandBrake rewrite does not add tags to the output. The lossless timestamp repair moves the FfMpeg `setts` output over the original with no MkvMerge pass. So it keeps the file's tags and attachments, and can add the tags FfMpeg writes. `RemoveTags` is the explicit removal, and it clears tags and attachments in place with MkvPropEdit. With it off, a file keeps its tags only while no MkvMerge pass rewrites it. The FfProbe and MediaInfo cover art paths clear attachments in place whatever `RemoveTags` says. The FfProbe path also forces `RemoveTags()`, as the cover art flow below shows.
Every step that rewrites the file writes its output through MkvMerge. That MkvMerge pass uses `MkvMergeBuilder.InputOptions.Default()`, which drops global tags, track tags, attachments, and buttons. That covers remux, track removal, re-encode, deinterlace, and closed caption removal. It also covers the lossless timestamp repair, repair by re-encode, and the cover art remux, the FfMpeg remux fallback included. The pass drops them whatever `RemoveTags` says, so an FfMpeg or HandBrake rewrite does not add tags to the output. `RemoveTags` is the explicit removal, and it clears tags and attachments in place with MkvPropEdit. With it off, a file keeps its tags only while no MkvMerge pass rewrites it. The FfProbe and MediaInfo cover art paths clear attachments in place whatever `RemoveTags` says. The FfProbe path also forces `RemoveTags()`, as the cover art flow below shows.

`Convert.ReMuxToMkv()` remuxes the whole file to the `.mkv` name, and a node that names it without a track selection runs this flow. With a track selection, it remuxes once with MkvMerge to `.tmp3`, with no FfMpeg fallback, then moves the output the same way.

Expand Down Expand Up @@ -791,7 +791,7 @@ flowchart TD
T0 -->|"fails on a cancel, a tool error, or a signal"| FAIL
T0 -->|"fails otherwise, warns, full ladder"| R0
T0 --> T1["TryLosslessTimestampRepair(): only when every non-monotonic stream is audio"]
T1 --> T2["FfMpeg rewrites timestamps to .tmp14"]
T1 --> T2["FfMpeg rewrites timestamps to .tmp14, then Convert.ReMux()"]
T2 --> T3["TimestampRepairRegressionGate(): stream hashes equal, start and duration within 40 ms"]
T3 --> T4{"Re-verify Clean?"}
T4 -->|yes| T5["Move over the original"]
Expand Down
1 change: 1 addition & 0 deletions HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Utility to optimize media files for Direct Play in Plex, Emby, Jellyfin, etc.
- Files an earlier version marked `VerifyFailed`, or marked `Verified` with the `Verify` option off, are not verified again automatically.
- Run the `createsidecar` command on the affected folders, which also clears the state that prevents a repeat conversion. The [`DtsTimestampRepair`](./Plugins/DtsTimestampRepair/) plugin re-verifies files marked `RepairFailed`.
- A non-monotonic DTS seen only after decode is repaired by remux only, and a file the remux cannot fix is marked `RepairFailed` and `PostDecodeTimestamps`.
- The lossless timestamp repair drops tags and attachments like every other rewrite, whatever `RemoveTags` says.
- An exception in one file fails only that file instead of ending the run, and the error summary names the failed step.
- `DeleteInvalidFiles` keeps a file after an exception, a cancel, a rename collision, or an unfinished tool run, and keeps a read-only or `PostDecodeTimestamps` file.
- With the `Verify` option off, a file is no longer marked `Verified`, and the sidecar records a clean full-file interlace or caption scan.
Expand Down
13 changes: 10 additions & 3 deletions PlexCleaner/FfMpegTool.cs
Original file line number Diff line number Diff line change
Expand Up @@ -452,15 +452,22 @@ public bool SetTimestamps(string inputName, string outputName)

public bool GetStreamHashes(string fileName, out Dictionary<int, string> streamHashes)
{
// Streamhash muxer hashes payload only, not timestamps, so an identical hash proves setts changed timestamps and nothing else
// Streamhash muxer hashes payload only, not timestamps, so an identical hash proves the repair left the media payload unchanged
streamHashes = [];

// Build command line, stream copy to the streamhash muxer written to stdout
// Stream copy the media streams to the streamhash muxer, the MkvMerge pass drops attachments and cover art
Command command = GetBuilder()
.GlobalOptions(options => options.Default())
.InputOptions(options => options.Default().InputFile(fileName))
.OutputOptions(options =>
options.MapAllCodecCopy().Format("streamhash").Add("-hash").Add("md5").Add("-")
options
.MapAllCodecCopy()
.Map("-0:t")
.Map("-0:disp:attached_pic")
.Format("streamhash")
.Add("-hash")
.Add("md5")
.Add("-")
)
.Build();

Expand Down
23 changes: 23 additions & 0 deletions PlexCleaner/FfMpegToolJsonSchema.cs
Original file line number Diff line number Diff line change
Expand Up @@ -50,13 +50,30 @@ public class StreamTimingsProbe
public static StreamTimingsProbe FromJson(string json) =>
JsonSerializer.Deserialize(json, FfMpegToolJsonContext.Default.StreamTimingsProbe)
?? throw new JsonException("Failed to deserialize StreamTimingsProbe");

// The MkvMerge pass drops attachments, cover art included, so only media streams are compared
public Dictionary<int, (double Start, double Duration)> MediaStreamTimings() =>
Streams
.Where(stream =>
stream.CodecType != "attachment" && !stream.Disposition.IsAttachedPic
)
.ToDictionary(
stream => stream.Index,
stream => (stream.StartTime, stream.Duration)
);
}

public class StreamTiming
{
[JsonPropertyName("index")]
public int Index { get; set; }

[JsonPropertyName("codec_type")]
public string CodecType { get; set; } = string.Empty;

[JsonPropertyName("disposition")]
public Disposition Disposition { get; } = new();

[JsonPropertyName("start_time")]
public double StartTime { get; set; } = double.NaN;

Expand Down Expand Up @@ -155,6 +172,12 @@ public class Disposition

[JsonIgnore]
public bool IsDescriptions => Descriptions != 0;

[JsonPropertyName("attached_pic")]
public int AttachedPic { get; set; }

[JsonIgnore]
public bool IsAttachedPic => AttachedPic != 0;
}

public class PacketInfo
Expand Down
13 changes: 6 additions & 7 deletions PlexCleaner/FfProbeTool.cs
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,9 @@ public bool GetStreamTimings(
.FfProbeOptions(options =>
options
.ShowStreams()
.ShowEntries("stream=index,start_time,duration")
.ShowEntries(
"stream=index,codec_type,start_time,duration:stream_disposition=attached_pic"
)
.OutputFormatJson()
.InputFile(fileName)
)
Expand All @@ -290,12 +292,9 @@ public bool GetStreamTimings(
// FromJson throws on malformed output
try
{
FfMpegToolJsonSchema.StreamTimingsProbe probe =
FfMpegToolJsonSchema.StreamTimingsProbe.FromJson(result.StandardOutput);
foreach (FfMpegToolJsonSchema.StreamTiming stream in probe.Streams)
{
timings[stream.Index] = (stream.StartTime, stream.Duration);
}
timings = FfMpegToolJsonSchema
.StreamTimingsProbe.FromJson(result.StandardOutput)
.MediaStreamTimings();
}
catch (Exception e) when (Log.Logger.LogAndHandle(e))
{
Expand Down
1 change: 0 additions & 1 deletion PlexCleaner/MkvMergeBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ public GlobalOptions Add(string option, bool escape)
public class InputOptions(ArgumentsBuilder argumentsBuilder)
{
// Drops tags and attachments whatever RemoveTags says, so a rewrite through MkvMerge never carries tool-added tags.
// ARCHITECTURE.md "Remux and Container Steps" names the rewrite that skips this pass.
public InputOptions Default() => NoGlobalTags().NoTrackTags().NoAttachments().NoButtons();

public InputOptions Identify(string option) => Add("--identify").Add($"\"{option}\"");
Expand Down
7 changes: 7 additions & 0 deletions PlexCleaner/ProcessFile.cs
Original file line number Diff line number Diff line change
Expand Up @@ -2614,6 +2614,13 @@ private bool TryLosslessTimestampRepair(DtsInfo? dtsInfo, out bool residualPostD
return false;
}

// The MkvMerge pass drops tags and attachments like every other rewrite, and the gate judges its output
if (!Convert.ReMux(tempName))
{
File.Delete(tempName);
return false;
}

// Reject unless the payload is byte-identical and the result verifies clean
if (!TimestampRepairRegressionGate(FileInfo.FullName, tempName))
{
Expand Down
21 changes: 21 additions & 0 deletions PlexCleanerTests/StreamTimingsProbeTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,25 @@ public void FromJson_MissingTiming_DefaultsToNaN()
_ = double.IsNaN(probe.Streams[0].StartTime).Should().BeTrue();
_ = double.IsNaN(probe.Streams[0].Duration).Should().BeTrue();
}

[Fact]
public void MediaStreamTimings_AttachmentAndCoverArt_Skipped()
{
// Lang=json
const string json = """
{ "streams": [
{ "index": 0, "codec_type": "video", "start_time": "0.000000" },
{ "index": 1, "codec_type": "audio", "start_time": "0.010000", "duration": "4.000000" },
{ "index": 2, "codec_type": "video", "disposition": { "attached_pic": 1 } },
{ "index": 3, "codec_type": "attachment", "start_time": "0.000000", "duration": "4.040000" }
] }
""";
Dictionary<int, (double Start, double Duration)> timings = FfMpegToolJsonSchema
.StreamTimingsProbe.FromJson(json)
.MediaStreamTimings();

_ = timings.Keys.Should().Equal(0, 1);
_ = timings[1].Start.Should().Be(0.01);
_ = timings[1].Duration.Should().Be(4.0);
}
}
2 changes: 2 additions & 0 deletions RegressionTests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ python3 synthetic/synthesize.py hdr10plus-cc -o hdr10plus-cc.mkv # true HDR1
python3 synthetic/synthesize.py dts-timebase -o dts-timebase.mkv # healthy VFR cadence, verifies clean
python3 synthetic/synthesize.py dts-duplicate-pts -o dts-dup.mkv # one repeated frame timestamp, fails verify
python3 synthetic/synthesize.py dts-mixed -o dts-mixed.mkv # audio DTS break plus post-decode video break
python3 synthetic/synthesize.py dts-audio-tagged -o dts-audio.mkv # audio DTS break under a tag and an attachment, repaired losslessly
```

### Targeting methodology
Expand All @@ -148,6 +149,7 @@ Pick the detections a fixture must reproduce, build a candidate, then **validate
- **Verify DTS time base** -- `dts-timebase` muxes a 23.976 cadence followed by a denser 29.97 cadence under a 23.976 default duration. Every timestamp increases at 1 ms precision, so verify passes it. A verify that encodes in the 1/framerate time base instead rounds the dense frames onto the same tick and reports a false non-monotonic DTS.
- **Verify DTS duplicate PTS** -- `dts-duplicate-pts` repeats one frame timestamp. Verify reports the repeat at native precision. The demuxer derives a DTS from the repeated timestamp, so the repeat is demux-visible and takes the full repair ladder.
- **Verify DTS mixed** -- `dts-mixed` swaps two B-frame video timestamps and steps one AAC packet timestamp back, using the unsorted mkvmerge v4 timestamps format. The demuxer derives video DTS from the sorted timestamps, so only the audio break is demux-visible, while the decoded video display order runs backward. The lossless `setts` tier repairs the audio but leaves the video failing verify, so the repair stops after the remux and marks the file `RepairFailed`. The v2 timestamps format cannot build this fixture, because mkvmerge sorts v2 timestamps and so removes both the swap and the step back.
- **Verify DTS audio tagged** -- `dts-audio-tagged` steps one AAC packet timestamp back under a global tag and an attachment, with healthy video. The lossless `setts` tier repairs the whole file. Its MkvMerge pass drops the tag, the attachment, and the tags FfMpeg writes, like every other rewrite. The regression gate compares only the media streams, since the attachment stream is gone after the pass.

### Extending the library

Expand Down
49 changes: 41 additions & 8 deletions RegressionTests/synthetic/synthesize.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
- DTS time base: a healthy cadence denser than the default duration, a false DTS only in a 1/framerate encoder time base.
- DTS duplicate PTS: one repeated frame timestamp, a real repeat that verify reports at native precision.
- DTS mixed: a demux-visible audio timestamp step back plus a B-frame video timestamp swap that only surfaces after decode.
- DTS audio tagged: the audio step back alone under a global tag and an attachment, which the lossless setts tier repairs.
"""

import argparse
Expand Down Expand Up @@ -215,18 +216,15 @@ def _count_packets(media: Path) -> int:
) # fmt: skip


def build_dts_mixed(out: Path) -> None:
"""B-frame video with two frame timestamps swapped, plus AAC audio with one packet timestamp stepping back.

The demuxer derives video DTS from the sorted timestamps, so only the decoded display order runs backward.
The audio step is demux-visible, and the lossless setts tier repairs it alone.
"""
def _build_dts_av(out: Path, swap_video: bool, extra: list[str]) -> None:
"""Mux B-frame video and AAC audio with one audio packet timestamp stepping back, optionally swapping two video frames."""
with tempfile.TemporaryDirectory() as td:
work = Path(td)
video, video_tc = work / "video.h264", work / "video.txt"
audio, audio_tc = work / "audio.aac", work / "audio.txt"
video_ts = [i * 1001 / 24 for i in range(96)]
video_ts[48], video_ts[49] = video_ts[49], video_ts[48]
if swap_video:
video_ts[48], video_ts[49] = video_ts[49], video_ts[48]
# The veryfast preset keeps B-frames, which the ultrafast default disables
gen_sdr_video(video, rate="24000/1001", frames=len(video_ts), preset="veryfast")
_write_timestamps(video_tc, video_ts, "v4")
Expand All @@ -235,12 +233,46 @@ def build_dts_mixed(out: Path) -> None:
audio_ts[100] = audio_ts[99] - 5
_write_timestamps(audio_tc, audio_ts, "v4")
run(
[MKVMERGE, "-o", str(out), "--default-duration", "0:24000/1001p",
[MKVMERGE, "-o", str(out), *extra, "--default-duration", "0:24000/1001p",
"--timestamps", f"0:{video_tc}", str(video),
"--timestamps", f"0:{audio_tc}", str(audio)]
) # fmt: skip


def build_dts_mixed(out: Path) -> None:
"""B-frame video with two frame timestamps swapped, plus AAC audio with one packet timestamp stepping back.

The demuxer derives video DTS from the sorted timestamps, so only the decoded display order runs backward.
The audio step is demux-visible, and the lossless setts tier repairs it alone.
"""
_build_dts_av(out, swap_video=True, extra=[])


def build_dts_audio_tagged(out: Path) -> None:
"""Healthy B-frame video plus AAC audio with one packet timestamp stepping back, under a global tag and an attachment.

The only DTS break is demux-visible audio, so the lossless setts tier repairs the whole file.
The tag and the attachment show whether the repaired file drops them like every other rewrite.
"""
with tempfile.TemporaryDirectory() as td:
tags, note = Path(td) / "tags.xml", Path(td) / "note.txt"
note.write_text("Synthetic attachment\n", encoding="utf-8")
tags.write_text(
'<?xml version="1.0"?>\n<Tags><Tag><Targets/><Simple><Name>COMMENT</Name>'
"<String>Synthetic</String></Simple></Tag></Tags>\n",
encoding="utf-8",
)
extra = [
"--global-tags",
str(tags),
"--attachment-mime-type",
"text/plain",
"--attach-file",
str(note),
]
_build_dts_av(out, swap_video=False, extra=extra)


TARGETS: dict[str, Callable[[Path], None]] = {
"hdr10-base": gen_hdr10_base,
"hdr10-multitrack": build_hdr10_multitrack,
Expand All @@ -249,6 +281,7 @@ def build_dts_mixed(out: Path) -> None:
"dts-timebase": build_dts_timebase,
"dts-duplicate-pts": build_dts_duplicate_pts,
"dts-mixed": build_dts_mixed,
"dts-audio-tagged": build_dts_audio_tagged,
}


Expand Down
Loading