Digging deeper: From micrograph curation to efficient particle picking

Particle picking can be challenging for several reasons, including low signal-to-noise ratio, sample heterogeneity, the presence of contaminants or non-vitreous ice, and the large number of micrographs and particles that need to be processed. When several rounds of particle cleanup are required, potentially useful particles can also be lost along the way.

In our CryoSPARC Case Study: End-to-End and Exploratory Processing of a Motor-Bound Nucleosome, several processing choices were used to reduce the time and particle-cleanup steps required to obtain a particle stack suitable for downstream processing and refinement. The strategy begins during preprocessing (Sections 1 to 4 of this case study), with early exploration and curation of the micrographs, making use of tools including Micrograph Junk Detector and Micrograph Denoiser, and beginning with an early inspection of the micrographs.

In this Digging Deeper post, we walk through the first four sections of the case study and examine the reasoning behind the processing decisions at each stage.

Work with a Subset of Micrographs

An important milestone in this workflow was obtaining a useful reference volume that could support more selective particle picking, for example with Template Picker. Generally, one option is to import an existing volume corresponding to the target protein and use it to generate templates. For a novel target or an unexplored protein complex, however, a reference can instead be generated directly from the dataset by progressing to Ab-Initio Reconstruction.

The dataset used in this case study is relatively large, comprising approximately 34,000 micrographs. Rather than carrying the complete dataset through the initial exploratory processing, a subset of approximately 5,000 micrographs was used for the first round of particle picking with Blob Picker and for the subsequent steps leading to template generation.

A subset for this type of exploration can also be generated in different ways. For example, Patch Motion Correction can be stopped after approximately 5,000 micrographs have been processed, or Exposure Sets Tools can be used to create a subset from the processed exposures. When computational resources are available, preprocessing of the remaining exposures can continue in parallel while the subset is used to work toward the first target reconstruction.

Get to Know Your Micrographs

The next step was to inspect the data and identify exposures that are unlikely to be useful for particle picking.

Here, Micrograph Junk Detector provided an early assessment of the types and amount of junk present in each micrograph. The job identifies features including intrinsic ice defects, such as non-vitreous ice; extrinsic ice defects, such as ice contamination; and carbon or gold regions when grid edges are visible in the movies. Its output includes micrographs with junk annotations, which can be inspected in Manually Curate Exposures.

When the input of the Manually Curate Exposures job are the labelled micrographs from the Micrograph Junk Detector, the annotations provide additional information for deciding which exposures to retain or exclude during manual curation.

Importantly, the input micrographs themselves are otherwise unchanged. The annotations can also be carried forward in the workflow and used to accept or reject particle picks based on whether they fall within regions identified as junk.

As thresholds were progressively applied in Manually Curate Exposures, the least promising micrographs were excluded from the dataset. In this case study, thresholds were set for CTF fit resolution, Relative Ice Thickness, Total full-frame motion distance, and Intrinsic Ice Defect Area, removing micrographs with less favorable characteristics before proceeding to particle curation and thereby eliminating obvious sources of junk.

As shown in the figure above, many of the same outlier micrographs are identified across metrics that capture different properties of the exposures. In the top-right panel, applying the selected thresholds also excludes many micrographs with elevated Junk Area (%), showing how the different indicators overlap for this dataset.

This provided a convenient way to inspect the micrographs while simultaneously identifying regions that should not contribute particles to the subsequent analysis.

Get to Know Your Particles

After the initial micrograph curation, the next step in the case study was to efficiently generate and clean an initial set of particle picks from a subset of 5,000 micrographs.

In this workflow, Micrograph Denoiser and Micrograph Junk Detector contribute complementary information to the initial picking: denoising facilitates identifying and picking particles, while the junk annotations identify regions from which picks should be excluded.

Running Micrograph Junk Detector on this curated subset also provided a useful view of the effect of the preceding micrograph curation. Compared with the distributions before manual curation, the Micrograph Junk Detector statistics reflect the removal of many of the least promising exposures, particularly those affected by intrinsic ice. This is consistent with the manual curation, during which thresholds for Relative Ice Thickness and Intrinsic Ice Defect Area were among those used to exclude micrographs.

At this stage, because the Blob Picker particles were also provided as input, Micrograph Junk Detector additionally output accepted and rejected particles. Picks located within regions classified as junk were automatically separated into the rejected output.

The accepted and rejected particle outputs from Micrograph Junk Detector could then be inspected with Inspect Particle Picks, bringing together the benefits of the preceding steps: Micrograph Denoiser facilitated particle visualization and picking, while Micrograph Junk Detector already removed picks associated with regions identified as junk.

The combined effects of Micrograph Denoiser, Blob Picker, and Micrograph Junk Detector were already apparent when the accepted particles from Micrograph Junk Detector are examined in Inspect Particle Picks. For this subset, the exclusion of picks associated with junk regions, together with the more clearly defined particle population obtained by picking on denoised micrographs, resulted in a more distinct NCC/power score distribution.

This facilitated the particle selection, either by setting thresholds manually or by using Auto Clustering, a mode designed specifically for particles picked from denoised micrographs.

The picks can be evaluated using their Normalized Cross Correlation (NCC) and power scores. NCC measures how closely a particle candidate matches the picking template in shape, whereas the power score reflects the signal intensity at the candidate location after background subtraction. In this case, the NCC/power score plot shows a main high-density cluster of picks, centered around a power score of approximately 100.

The particles could be extracted applying Fourier cropping, in this case to a box size of 64 pixels, saving disk space and allowing the subsequent jobs to run faster. Since the aim of this initial processing stage was to generate templates for Template Picking, which would be filtered to 20 Å, the high-resolution information removed by Fourier cropping was not needed at this stage.

At this point, the initial goal of working with the 5,000-micrograph subset had been reached: the resulting particle set was sufficient to generate the first useful reconstruction of the target complex as follows.

Create Your Templates

The selected particles were first subjected to 2D Classification, followed by a decoy-classification strategy to further clean the particle stack. To generate the target reference volume, Ab-Initio Reconstruction was performed using a subset of 20k particles from the selected 2D classes. In parallel, 1k particles from the excluded classes were used to rapidly generate three junk volumes, with the resolution and number of iterations limited to accelerate their reconstruction.

The resulting target and junk volumes were then used as competing references in Heterogeneous Refinement, using all particles from the selected 2D classes as input. In this case study, this decoy-classification step allowed Heterogeneous Refinement to distribute the particles among the target and junk references, further separating unwanted particles while retaining a cleaner particle population in the target class, including less abundant views.

Once the good particles are classified in the target class of the decoy classification, 2D Classification and Select 2D Classes were used to create a good set of templates.

In this case study, working first with a subset of micrographs provided a faster route to the initial target reconstruction and templates required for more selective picking across the full dataset. Along the way, Micrograph Denoiser and Micrograph Junk Detector contributed complementary information: denoising facilitated particle visualization and Blob picking, while junk annotations supported both exposure curation and the exclusion of picks from undesirable regions. Finally, decoy classification provided an alternative to repeated rounds of 2D Classification and visual particle selection, using Heterogeneous Refinement to distribute particles among the target and junk references and further clean the particle stack while retaining less abundant views.

With these templates and the initial curation strategy established, processing could return to the full dataset for Template Picking on denoised micrographs, again incorporating the Micrograph Junk Detector annotations.