Skip to content

Error while parsing partially saved nd2 files #23

Open
Open
Error while parsing partially saved nd2 files#23
Labels
enhancementhelp wanted

Description

@bellonet
bellonet
opened on Sep 18, 2019

The reader works well for most nd2 files, however some of my nd2 files are not parsed correctly and I get an error:
EmptyFileError: No axes were found for this .nd2 file.

I think the error occurs during parsing from the end of the file (in _build_label_map):

# go 8 bytes back from file end
self._fh.seek(-8, 2) 
chunk_map_start_location = struct.unpack("Q", self._fh.read(8))[0]

where for the files that fail: chunk_map_start_location is assigned to 0.

I suspect that this happens with nd2 files that had an error during acquisition, so the end of the file might be corrupt. However, since nd2 files hold multiple images it means that there are many valuable images in the same file.

The bioimage reader in ImageJ parses and opens those (maybe corrupt) files correctly.
I will be happy to send you an example file, the smallest I have that fails is ~8GB.

Activity

  1. rbnvrw commented on Sep 19, 2019

    @rbnvrw
    rbnvrw
    on Sep 19, 2019
    Member

    Dear @bellonet thank you for reporting this.
    Indeed, the parser now looks for 8 bytes at the end of the file that indicate where the information of frames is stored in the ND2 file. If this is missing, the file cannot be parsed correctly.

    Because there is no description of the 'standard' ND2 format, it is non-trivial to fix this issue.
    The code in this repository was created by reverse engineering the ND2 file format.

    However, because it is apparently working with the Bioformats plugin for ImageJ, it can be done.
    I will leave this issue open for now but I do not have time at the moment to provide a working implementation. If anyone has time to look at this, PRs would be greatly appreciated.

  2. rbnvrw
    added
    enhancement
    help wanted
    on Sep 19, 2019
  3. rbnvrw
    changed the title [-]Error while parsing some nd2 files[/-] [+]Error while parsing partially saved nd2 files[/+] on Sep 19, 2019

  4. ggirelli commented on Aug 13, 2020

    @ggirelli
    ggirelli
    on Aug 13, 2020
    Contributor

    Hi, just dropping a line here in case it might help. I have had some experience with broken nd2 files when the file transfer from the server got interrupted. It is true that the nd2 file is opened by ImageJ (or FIJI), but at least for my cases the single stacks where all jumbled up together. I.e., the beginning of the second was in the end of the first and so on, or viceversa. I am not sure whether it is worth investing time in rescuing these files without Nikon's help (imho).

  5. lillyguisnet commented on Aug 10, 2021

    @lillyguisnet
    lillyguisnet
    on Aug 10, 2021

    Hi! Just wondering if there has been any updates on this issue or plans to address it in the future? Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementhelp wanted

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions