Establish a way to precisely define time points in an experiment

[eric-czech]

Timestamps in image files are ok for inferring time points (usually as hours), but they are less helpful in images from multiple experiments (starting at different times) and with images where the timestamps themselves may not be trustworthy enough to simply take the minimum one as a starting point. It may be worth adding mappings of dates/times to time points in metadata.

[yellenlab]

Hi @eric-czech

Thanks for making all these recent changes. I really like the new app, and am already using it a whole bunch. Just a few suggestions, some quick fixes and other more long term changes, that could make it work way better for us…

1. One of the features that we will be using very frequently is the apartment and array functions at the t=0 time point. Since it takes about 30 minutes to load and image each chip, this t=0 time point will not be the same across the entire experiment, and in fact is unique to each chip and designated as the first time point that the chip was imaged. We would like to know 2 things for this t0 time point. First, how many cells are in the component_trap, and how many cells are in the component_chamber. For this, we may need to adjust the chip-config so that the images show both the trap and chamber. This is something we can easily do in the chip-XX.yaml files. We would like to easily extract the following quantities: i) what % of apartments have 0, 1, 2, 3, 4, … cells in the component_chamber at t0, and ii) what % of apartments have 0, 1, 2, 3, 4, … cells in the component_trap at t0.

2. Another thing we are gearing up for is the ability to take fluorescent images of cells and do overlays. We aim to do this for multiple reasons. For example, in some experiments, we will be including a mixture of GFP-labeled drug resistant cells and non-labeled drug sensitive cells. In another experiment, we aim to demonstrate the ability to form pairs of T-cells and cancer cells for cancer immunology applications. We will of course need to do annotations for the fluorescent images, however I just wanted to make you aware of this in case there is anything you can do behind the scenes to get ready for it. Also, if you have any instructions on how we can do it to interface best with your end, that would be super.

3. There are also minor issues with the cells not being correctly identified in some images, particularly when they are densely packed, even at 20% confluent. I can upload a few datasets where this happened frequently. It is probably an issue of us not getting the best focal plane, or perhaps we need more images to annotate. This also makes me wonder if we should go back to the idea of determining the clump area in addition to identifying the single cells, as the union of these two would be the best indicator that everything is working correctly.

Otherwise, maybe we can chat about all this when you finish up with your other responsibilities.

Thanks again for the help!

@benjaminyellen, @jmotschman

[eric-czech]

Hey Ben,

Some thoughts on those:

1. Should be straightforward enough to surface those (they're already in the data but I'll make it a point to add them in the app)

2. Makes sense -- I think the biggest question from here is how those images are captured. Can they be stored as a single tif file with the brightfield and fluorescent channels or will it be one brightfield image and one or more fluorescent images (possibly with slight spatial shifts)? The former is certainly easier on my side of things, and easier to keep organized too, but I can imagine that might be tougher on the scope automation side. I think this is the ideal scenario though, let me know if you disagree:

   • Scope captures all channels at one position at nearly the same time
   • Models are used only to determine cell boundaries from brightfield channel
   • Fluorescent intensities are summed or averaged over the area of a cell (or possibly individual parts of a cell if you ever wanted to get into fluorescent stains for morphology)
   • Cells are classified based on fluorescent intensity after the fact instead of by a learning model

3. I think we'll be able to get rid of the large majority of the misclassifications with heavier augmentation and low amounts of annotation with more diversity. I think the simplest way to get started on that is just to start saving those cases you guys find so we can make sure they end up getting annotated. Even screenshots from the app would work, though I added Add image exporter from app  #44 and will make that a top priority so it's easy to get images at their original size (they might be originals now though -- I'll have to check). As for the cell clump, I think the problem with that is that the model should essentially learn to make cell clump predictions based on the same conv filters it uses for cells (so they're both untrustworthy). Perhaps a good sanity check instead would be a threshold image confined to the chamber in the chip config ... the positive pixels from that should be a good independent estimate of how much area the cells take up (added issue: Add sanity check statistic for cell area segmented #45).

[yellenlab]

Also, before I forget, we are now using two types of alignment marks, so that we can do orientation as well as xy positioning. I'm guessing it might be a little complicated to modify your code to handle these two alignment marks, however if you think it's possible and/or if you recommend a different type of alignment mark to get the orientation correct, then we can easily modify future designs to do that. Let me know if you have any suggestions.

[yellenlab]

Regarding the multi-channel capture issues, currently we take a whole set of brightfield images, followed by a whole set of fluorescent images. We do this because the filter cube is a mechanical component, and we would literally have to switch it 6000 times per experiment, which would rapidly wear it out. However, perhaps there is a simple method, such as if the time elapsed between the BF and GFP image, for example, is less than 30 minutes, then for practical purposes they can be considered as derived from the same image time point. I recognize that this limits us in terms of analyzing fast biological processes, but there is always going to be a tradeoff between large numbers of measurements vs. time of measurement.

[yellenlab]

Also, I agree with your idea of using the brightfield image as the mask, which obviates the need for a training model just for the fluorescent images. This is a great idea.

[eric-czech]

Breaking this out into issues on:

• Fluorescent Image Support
• Multiple Alignment Markers

As far as the original issue goes it looks like there's not much to do then. The code currently uses the first timestamp found for each array since chip identifiers aren't mandatory in the experiment configurations, but that should suffice. I added #53 though to make sure there is eventually some way to see what time zero is for each array as well as all other hourly buckets.