1 T20S Tutorial
Welcome to cryoSPARC v2. We recommend processing the tutorial dataset first to become familiar with the workflow in cryoSPARC v2. The dataset is a subset of 20 movies from the EMPIAR-10025 T20S Proteasome dataset. If you follow the instructions below, you will learn how to use most of the common job types in cryoSPARC, and you should be able to achieve a 3.0A refined structure with this dataset.
1.1 Dashboard, Projects, Workspaces and Jobs
The Dashboard provides at-a-glance information on your Projects, Workspaces and status of Jobs, as well as a change log documenting updates in new versions of cryoSPARC. The header and footer contain links to Projects view, Workspaces, the Resource Manager and the identity of the current user.
CryoSPARC organizes your workflow by Project, e.g, P1, P2, etc. Projects contain one or more Workspaces, which in turn house Jobs. Projects are strict divisions, in the sense that files and jobs from different projects are stored in different project directories, and jobs cannot be connected from one project to another. Workspaces, on the other hand, are intended simply to allow logical separation of jobs and workflows so that they can be more easily managed in a large project. Jobs can be connected across workspaces, and each job can belong to more than one workspace.
1.2 Create a Project
- Navigate to the Projects view by clicking on the drawer icon on the header, or from the Projects button in the footer.
- To create a project, press
Nor click on the "+" on the header, which will bring up a modal window for the New Project details.
- Enter a project Title and select a location for the associated project directory from the File Browser. The project directory you select should already exist, and it will be populated with job directories as you create jobs. All files associated with the project will be stored inside the selected project directory. You may also wish to enter a Description for your project.
- Your new project now appears on the Projects page.
1.3 Create a Workspace
Workspaces are meant to organize or separate portions of the cryo-EM workflow for convenience. You will need to create at least one Workspace within a Project before you can run a Job.
- Click the project number (e.g., P8) to open the Project within which you want to create a Workspace.
- Alternatively, you can open a Project by clicking on the Projects drop-down on the header. This will bring up a searchable list of all Projects associated with your user account. Clicking on a Project will open that Project.
- Once inside your selected Project, create a New Workspace using the "+" button on the header or
Non your keyboard. Workspace titles can be changed later, and descriptions can be added any time.
1.4 Download tutorial dataset
- On the machine where you have cryoSPARC installed, use the terminal to navigate to the directory where you would like the test dataset do be downloaded. It is approximately 8 GB. The location where you download the test data should be readable by the user that is running cryoSPARC processes.
- Run the command
cryosparcm downloadtestin this directory. This will download a subset of the T20S dataset.
tar -xf empiar_10025_subset.tarto decompress the downloaded data.
1.5 Import movies
In cryoSPARC, navigate to the new Workspace you created above. To do so, navigate to the project to see a list of workspaces, and then into the workspace.
Click the pickaxe icon on the right sidebar to display the Job Builder. The Job Builder displays all available job types by category (e.g., workflows, imports, motion correction, etc.)
Select the Import Movies job type in the Job builder. A new job is created within the Workspace, displayed as a card. By default, new jobs are set to Building status, indicated on the job card in purple. To change paramaters, you can toggle between active or inactive building states by pressing
Bon your keyboard, or clicking on the Building button on the job card. (See also 'The Basics' portion of the Guide for more keyboard shortcuts.)
Select the Movies data path by clicking on the file browse icon and select the movie files (.mrc or .tif). To select multiple files, use a wildcard, e.g.,
*.mrc. This will select all files that match the wildcard expression, and the list of selected files will be displayed in the file browser, along with the number of matches at the bottom. In the case of the tutorial dataset, navigate to the location where you downloaded the test data, and use the wildcard expression
*.tifto select all TIFF format movies in the folder. There should be 20 movies that are imported.
Select the Gain reference path also by using the file browser. In this case, select the single
.mrcfile in the folder where the test data was downloaded.
Job parameters can be edited directly from the Builder. Input the following parameters (obtained from the original publication in eLife). Raw pixel size (A): 0.6575 Accelerating voltage (kV): 300 Spherical abberation (mm): 2.7 Total exposure dose (e/A^2): 53
When you edit a parameter, the blue 'D' icon will vanish indicating the parameter was changed from its default value, and the parameter will be highlighted in green:
Click Queue to start the import. A modal appears allowing you to select a lane/node on which to run the job. The available lanes will depend on your installation configuration. Once you have selected a lane, click Create.
The import job will queue, then start running. You can monitor the status of the job by looking at the card for that job.
To open a Job and view its progress, you can click on the Job number on the top left hand side of the Job card, or you can click on the job card and press the
This will bring up the inspect view, which is comprised of a stream log that displays real-time progress of the Import Job. You can scroll through the stream log to view results, and select a checkpoint to find a specific location in the stream log, or 'Show from top' to return to the beginning. The inspect view also shows all outputs of the job on the right hand side:
To exit the job/close the inspect view, you can press the
SPACEBARagain, or click the
xon the top right side of the inspect view.
Once completed, the job's status indicator and status on the Job Details tab will display in green. Other detailed information about the job, along with actions that can be performed, are also listed in the details panel. The Output of the import job, i.e., the 20 imported movies, are available on the right hand side of the stream log.
1.6 Motion correction
In the Job Builder, select Full-frame motion correction (1 GPU) or Full-frame motion correction multi (parallelized over multiple GPUs, if you have them available). This will create a new job and place it in building state so that it's inputs and parameters are editable in the right side panel.
The motion correction job requires raw movies as Inputs. Open the previously completed Import Movies job, then drag and drop the Outputs of the Import Movies job, to the Movies placeholder in the Job Builder.
Once dropped, you will see the connected output name appear in the Job Builder as an Input:
Queue the job and select the appropriate lane in which to run it. The rigid motion correction job parameters generally do not need to be adjusted and automatically tune based on the data.
1.7 CTF estimation
CryoSPARC v2 provides a wrapper to CTFFIND4 for robust CTF estimation of micrographs after rigid motion correction. (Alexis Rohou and Nikolaus Grigorieff. CTFFIND4: Fast and accurate defocus estimation from electron micrographs. Journal of Structural Biology, 192(2):216–221, November 2015). Please see the CTFFIND4 License Terms at the end of this section.
Select CTF Estimation in the Job Builder to create a new job.
This job type requires micrographs as the Input. Open the previously completed full-frame motion correction job, and drag and drop the Output (20 micrographs) into the Micrograph placeholder in the Job Builder. Queue the job to start.
CTFFIND4 License Terms
The Janelia Research Campus Software License 1.2
Copyright (c) 2018, Howard Hughes Medical Institute, All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
- Neither the name of the Howard Hughes Medical Institute nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT, OR FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; REASONABLE ROYALTIES; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
1.8 Particle picking (interactive)
We start off by manually picking some particles from the previously processed micrographs to generate templates for further automatic picking. Manual picking is a good idea because it ensures that users are aware of the quality of their data and what to expect particle images, projections, and structures to look like. This helps avoid scenarios where automatic picking algorithms lead to incorrect results.
Select Manual picker from the Job Builder.
Locate the micrographs_success output from the previously completed CTF estimation job, and drag and drop these into the Micrograph placeholder in the Job Builder. Queue the job.
This interactive job type displays all micrographs used as inputs on the left hand side. Click on 'Name', 'Defocus', 'CTF fit' or 'Picks' to re-order the list.
To pick from a particular micrograph, locate that micrograph in the list and click on it (purple highlight).
Box size: We generally recommend selecting a box size that is at least double the diameter of the particle. In this case, enter a box size of 384. The box size controls how much of the micrograph is cropped around each particle location, and larger sizes help to capture the most high-resolution signal that is spread out spatially due to the effect of defocus (CTF) in the microscope. Larger box sizes do, however, significantly increase computation expense in further processing.
Hover over a particle and left-click to select. Once selected, it will be surrounded by a green circle indicating the box size. The number of picked particles in each micrograph will display in the 'Picks' column and the total number of picks is available at the bottom of the list. For this tutorial, we recommend picking a total of approximately 100 particles across the 20 micrographs. Try to capture a diversity of top and side views of the Proteasome.
To un-pick a particle, right click over it.
Your picks are automatically saved. Closing the browser also does not affect picks.
You can use the lowpass filter slider on the left side to adjust the display of the micrograph (this does not affect the results of the job) and/or adjust the box size if needed.
Once you are satisfied with your picks, click Done Picking! Extract Particles at the top. The job will proceed to extract the particles you picked manually.
1.9 Template-based automatic particle picking
- To generate templates for automatic picking, select 2D Classification from the Job Builder. Drag and drop the particles output from the previously completed manual picking job, into the Particle stacks input.
- Modify the Number of 2D classes parameter to 10, then Queue the job.
- The 2D Classification job will proceed through a number of iterations, and should take a minute or two. Once complete, create a Select 2D job from the Job Builder. Drag and drop both the particles and class_averages to their corresponding Inputs in the Job Builder and Queue the Select 2D job.
- Once ready, the Select 2D job will enter Waiting mode (fuschia). Waiting mode indicates that an interactive job is ready for a user to interact with, but opening the inspect page (
SPACEBARor click the job number of the job).
- Select two templates (one top view and one side view) by clicking on the respective 2D classes. Then, click Done to complete the job.
- From the Job Builder, create a Template Picker job. This job will use the templates generated to automatically pick particles across all micrographs.
- Drag and drop the templates_selected (2) from the previously completed Select 2D job, into the Templates input.
- Locate and open the previously completed CTF Estimation job. Drag and drop the micrographs_success (20) into the Micrographs input.
- Set the Particle diameter in Angstrom parameter to 190. Then, Queue the job. The template picker will quickly pick out particles that match the templates and return statistics for each pick that can be used to filter high quality picks.
1.10 Inspect picks
The Inspect picks job allows you to view and interactively adjust the results of template based automatic particle picking.
- Select Inspect Picks from the Job Builder.
- Drag and drop both the particles and micrographs outputs from the previously completed Template Picker job. Queue the job.
- You can browse through the list of micrographs on the left side.
- The histogram on the left side shows statistics across all pick locations, including false positives and true particles. True particles generally have a high NCC score (indicating agreement in shape with the templates) and a high Power score (indicating the presence of signficant signal). Picks that have too little Power are false positives containing only ice, while picks with very high power are carbon edges, ice crystals, aggregate particles, etc.
Make adjustments to the parameters below if needed:
- Adjust the lowpass filter slider if needed to better view the picks.
- Adjust the box size to make it easier to see the location of picks. Often a very small box size (32) can be helpful. The box size is not used in the outputs of this job, as this job only outputs particle locations.
- Adjust the Normalized cross correlation (NCC) slider (approx. 0.350).
- Adjust the Power threshold slider. This helps to remove false positives (approx between 1075 and 1745).
- Your adjustments will be saved automatically.
- Once you are satisfied with your picks, click Done Picking! Output Locations!. This will complete the Inspect picks job.
1.11 Local motion correction
Local motion correction estimates and corrects for anisotropic local beam-induced motion on a per-particle basis, from movie data and a pre-estimated full-frame motion trajectory.
- Select Local motion correction (1 GPU) or Local motion correction (multi) (parallelized over multiple GPUs, if you have them available).
- Open the previously completed Inspect Picks job. Drag and drop both the micrographs and particles outputs into the corresponding inputs on the Job Builder.
- In the Particle Extraction section of the Job Builder, change the Extraction box size (pix) to 440. Note that the local motion correction job is responsible for extracting particles, accounting for radiation damage, and correcting for local motion. In this case, we use an extra large box size for the final particles (as opposed to the smaller box size used when we manually picked particles) with the hope of achieving a high-resolution reconstruction.
- Queue the job. The stream log will display local motion trajectories and a plot of exposure weights for each micrograph being processed.
1.12 2D Classification
- Select 2D Classification from the Job Builder.
- Drag and drop the particles output from the previously completed Local motion correction job, into the input.
- Queue the job.
- In the stream log, you will see images of class averages during every iteration. Classification into 50 classes (the default number of classes) should take about 15 minutes.
- Once complete, proceed to Select 2D classes.
1.13 Select 2D classes (interactive)
Select Select 2D classes from the Job Builder.
Drag and drop both the particles and class_averages that were outputs of the previously completed 2D Classification job.
Queue the job. Once the data has loaded, the job status will change to Waiting to indicate you may now proceed to select the classes you wish to keep.
Click on each "good" class to select it. You can use both the number of particles and the provided class resolution score to identify good classes of particles. There are several ways to sort the classes in ascending or descending order based on:
# of particles: The total number of particles in each class
Resolution: The relative resolution of all particles in the class (Å)
ECA: Effective classes assigned
Note: Avoid selecting classes that contain only a partial particle or a non-particle junk image.
When finished, click Done at the top right side of the window. The job will complete.
1.14 Ab-initio reconstruction
- Select Ab-initio reconstruction from the Job Builder.
- Drag and drop the particles_selected output from the previously completed Select 2D classes job, into the Particle stacks input in the Job Builder.
- Queue the job. Results will appear in real time in the stream log as the iterations progress. Ab-initio reconstruction should be able to resolve the T20S structure to a coarse resolution.
1.15 Homogeneous refinement
- Select Homogeneous refinement from the Job Builder.
- Drag and drop both the particles_all_classes and volume_class_0 , from the previously completed ab-initio reconstruction, into the Particle Stacks and Initial Volume inputs, respectively.
- Set the following parameters: Refinement box size: 256 Symmetry: D7
- Queue the job. Results will appear in real time in the stream log. The refinement job will perform a rapid gold-standard refinement using the branch-and-bound algorithm. At each iteration, the resolution and other diagnostic information is displayed.
- Once complete, you can download the volume and/or mask directly from the Outputs section on the right hand side. Click on the drop-down to select the outputs you wish to download. For a refinement job, the output called
map_sharpis the final result with automatic B-factor sharpening applied, filtered to the estimated FSC resolution. Note that for optimal results in publications and for model-building, often re-sharpening and adjustment of the B-factor is necessary.
- You can sharpen the result of the refinement using the Sharpening Tools from the Utilities section of the Job Builder.
- Drag and drop the volume output from the result of the previous refinement job, into the input volume input.
- You will need to input a B-Factor. The best place to start is to obtain the B-Factor from the final Guinier plot (available in the streamlog of the refinement job you previously ran). Ensure you input the B-Factor as a negative value.
- Queue the job.
- Once complete, you can download the sharpened map from the output.
- After visually assessing the map, you may wish to run another sharpening job with a different B-Factor.
1.17 Inspecting Workflows
Once you have assembled a workflow of connected jobs within a project, it can be helpful to view the tree of jobs and output connections to understand how a result was obtained. This can be done simply by switching to the Tree view while within a project or workspace. To open the Tree view, click on the flowchart icon in the header:
Within the tree view, jobs can be selected and modified/connected the same way as in the Card view shown above. For more information on the Tree view and other useful tips, see the Basics.
1.18 Further processing
Now that you have refined the data to a high-resolution structure, more advanced processing techniques can be applied. Explore the job builder and other documentation to see the available job types and processing options. Common workflows include:
- Multiple rounds of 2D classification to remove more junk particles
- Heterogeneous ab-initio reconstruction to find multiple unexpected conformational states or multiple distinct particles in the data
- Heterogeneous refinement to refine multiple conformations and simultaneously classify particles
- Sub-classification to identify small slightly differing populations
- Non-uniform refinement to account for disordered regions and local variations in a structure
- Masked/local refinements to focus on sub-regions of a structure
- Re-picking with multiple higher quality 2D classes
- Local or per-particle CTF re-estimation
For detailed explanations of all available job types and commonly adjusted parameters, see Guide to Jobs and Common Parameters. Also check back to see updates to this guide, as new features and algorithms are in constant development within cryoSPARC.