Next: 2 B. Anatomical Image Up: 1 A. Overview Previous: 5. Looking At Images Contents

Subsections

6. Advanced Image Visualization

**Figure 6.1:** Displaying Image Sequences. When a 4D image is loaded, two additional controls appear in all viewers. (i) The Frame Scale which allows the user to change frames and (ii) A ``Save All'' button which takes a series of snapshot of the current viewer - one for each frame. ( This is equivalent to acquiring a series of snapshots manually using the ``Save'' button next to it).

6.1 4D Images

Often medical images are acquired in temporal sequences over time. Common examples of this are fMRI T2* time series and cardiac imaging.

BioImage Suite can happily handle 4D image display and manipulation. In fact, BioImage Suite started out as a package for cardiac image analysis, so the functionality for 4D images was there from the beginning (e.g. the 4D version of the SurfaceEditor predates the 3D version by about 7 years!). When a 4D Image is loaded in a BioImage Suite viewer, a couple of extra controls appear to facilitate manipulation of this type of images, as shown in figure 6.1.

In addition, BioImage Suite has a specialized viewer that can be used to ``play'' movies (or cine-loops) of 4D images. Applications that use this viewer can be found under the ``Cardiac'' tab of the BioImage Suite main menu. Figure 6.2 shows a snapshot of the ``VolumeViewer'' application. While this application, has some specialized functionality for Cardiac Image processing (to be found under the ``Cardiac'' menu), it can also be used to play movies of fMRI timeseries data. This can be useful for checking motion correction, intensity stability etc.

6.2 3D Rendering Controls

The orthogonal viewer has four sub-viewers. Three of these mimic 2D displays and are used to show one of the three orthogonal views ``from above'' e.g. Axial slice, and have stringent controls on where the camera can move (i.e. it is forced to stay in the same plane). The fourth viewer (the 3D Renderer) allows the user to completely control the camera position and to interact with the images (and surfaces, electrodes, landmarks etc) in 3D. There are two aspects of this renderer that we will discuss: (i) how the images are displayed - see Figure 6.3 - e.g. enabling/disabling volume rendering etc and (ii) the camera controls.

**Figure 6.2:** The specialized 4D viewer. One additional button (marked as A) called ``Movie Control'' is present. Pressing this shows the ``Movie Controls'' dialog which is shown on top of the main viewer. Within this, there are standard facilities for playing movies. The speed of the movie (in frames/second) is set using the ``Rate'' scale (B). This is maximum speed to be used and may not be achievable depending on the hardware. There are two modes of playing movies (a relic of older, slower graphic cards). The mode is selected by the ``Play Mode'' option menu (marked as C) and can be either ``Complete'' or ``Fast''. Using ``Complete'' mode will result in slower performance - but the full viewer facilities (e.g. changing slice etc.) are available during the movie playing. Using ``Fast'' (as the name implies) will result in faster performance as BioImage Suite will cache all display prior to playing the movie. **Before** playing a movie in ``Fast'' mode use the ``Prepare'' button (marked as D) to cache all frames.

**Figure 6.3:** The orthogonal viewer has two option menus for controlling what gets displayed in the 3D Renderer. These are highlighted in this figure. The first control selects which of the combinations of possible 3D image displays are used (none, 3-slice display, volume rendering, oblique slice). The second selects the ``decoration'' around the image. ``Box'' refers to a cube placed around to show the outline (extent) of the image, whereas ``axis'' shows the native axes of the image (Red for the i-axis, Green for the j-axis and Blue for the k-axis - the slice selection).

6.2.0.0.1 The Camera Controls:

In 3D Mode the user is free to move the camera as they please around the viewer. In particular, they can manipulate the camera position, rotation, zoom and clip planes. In addition, six preset camera positions are available and the user can also program (and save) other presets as needed. The camera controls or ``3D Renderer Controls'' can be accessed by pressing ``Shift + Right Mouse Button'' anywhere in the 3D Viewer. The available controls are explained in more detail in Figure 6.4. In addition, the left mouse button is used to rotate the camera, the middle mouse button (press the wheel!) is used to zoom in and out and right mouse button is used to translate the camera.

**Figure 6.4:** **The 3D Renderer Controls**. **Top Left:** The 3D Renderer Control Windows. This has 4 panels shown in zoom-ups below.
Panel 1: Camera Zoom and Rotation. There are three controls for setting camera zoom and rotation. A: The Zoom Control. B: The azimuth rotation control and C: The elevation rotation control. The amount of rotation is set by the drop menu between the two rotation controls. Panel 2: Camera Position Presets: The drop menu (D) allows the user to select the preset camera position (thing of these as camera ``bookmarks''). There are six positions by default. Additional positions can be set using the ``Add'' button (E). A current preset may be updated with the camera position using the ``Upd'' button (F). Camera presets can be loaded and saved to file using the ``Load'' and ``Save'' buttons respectively. (G). Panel 3: Camera Clipping: There are two controls here for setting the near clip plane and the clip thickness. Objects nearer to the camera than ``Near Plane'' or further away than ``Near Plane+Thickness'' are eliminated from the scene. These are restored to their default values using the ``Rt'' (Reset button) highlighted as H in Panel 4. Panel 4: Polygonal Clipping: In addition to the general camera clipping controls (Panel 3), Bioimage Suite has additional clipping controls to ``cut'' either volumes or surfaces separately in planes that are not perpendicular to the camera. The volume clipping controls are on the Volume Control. The controls in this panel can be used to clip surface objects. The clipping controls are enabled using the ``Enable Polygonal Clipping'' checkbox (I). Six scales are provided to crop in each axis (J).

6.3 Volume Rendering

6.3.1 Direct volume rendering

Volume rendering is defined as the process of generating a 2D image directly from three-dimensional data. The basic idea is to simulate the absorption and emission of light as it goes through matter.

To simulate the passage of light through data, rays are cast from the image plane into the space containing the volume, as shown in Figure 6.5. For every pixel in the image plane, a ray is cast into the volume space that traverses the volume. At each point along the ray, the data is sampled to identify the density/intensity at that point. This value is different for different materials such as bone, tissue, fat and so on. Based on the intensity at that point along the ray, the color and opacity is identified using a lookup table called a colormap or transfer function.

6.3.2 Types of compositing functions

Color and opacity are accumulated along the ray and various compositing operators are used to get different effects. An X-ray image can be simulated by averaging the intensity values sampled along the ray. A MIP (Maximum intensity projection) image is obtained by preserving the maximum intensity value along the ray. MIP images are frequently used to visualize vascular structures as they can be clearly seen using this technique. Figure 6.6 shows an example of the two techniques.

**Figure 6.5:** This image depicts a schematic of the volume rendering process. For every pixel in the image plane, a ray is cast into the volume space. Along the ray, the volume rendering integral is evaluated to simulate the passage of light through matter. At each position along the ray, the volume/data is sampled to identify the value at that location. The location identifies the material properties such as absorption and emission. A color and opacity is obtained for every density by performing a lookup into a table, called the *colormap or transfer function*. This allows users to color different regions of the volume differently.

**Figure 6.6:** The left image shows an X-ray style image that can be generated by average the intensity values along the ray. The right image shows a MIP image that can be obtained by picking the maximum intensity observed along that ray. Such MIP images are widely used to visualize vessels.

The compositing technique used for direct volume rendering is based on accumulating the color based on the color and opacity of the current voxel under consideration. For example, during ray traversal if in a CT scan, bone is encountered, it absorbs more energy than tissue or a thin vessel.

6.3.3 Types of Volume Rendering

There are four types of volume rendering techniques: raycasting, texture mapping-based, shear warp and splatting. Raycasting and texture mapping-based techniques are the most widely used of the four. Raycasting is the technique in which an image is generated by casting rays into the volume space.

As graphics hardware got better with time, researchers devised a way to perform volume rendering using graphics hardware. This technique loads the data into graphics memory and utilizes 2D (and more recently 3D) texture maps to perform interpolation and blending. Since graphics hardware is extremely fast, this technique is much faster than the raycasting technique described earlier.

6.3.4 Volume Rendering Facilities in BioImage Suite

BioImage Suite has support for three types of volume rendering namely (a) software based raycasting, (b) texture mapping accelerated rendering and (c) software maximum intensity projection. Each has their own applications. For most studies using reasonable hardware option (b) is optimal (and hence the default).

In the case of texture mapped volumes, it is worth noting that the underlying graphics hardware processes images only in dimensions of powers of 2, e.g. 64, 128, 256 etc. This means that a 65x65 image is as computationally expensive to render as an 128x128 image, since at each stage the rendering engine zero-pads the image size to the next power of 2! BioImage Suite reslices images prior to rendering to specific sizes to optimize this process. In particular note that cropping the volume (using the ``Volume'' control, shown in Figure 6.7 - H) can dramatically improve the quality of the rendering, as all the rendering pixels (e.g. 64x64) are allocated for the cropped portion as opposed to the whole image.

One more key point is to note that the volume rendering facilities have a separate colormap from the one used for slice display. Some synchronization is present though. For example, when preset colormaps are selected in the main viewer (e.g. St or Nr) their equivalents are also automatically applied for volume rendering.

**Figure 6.7:** **Volume Rendering Controls.**. A: Show volume, if unchecked the volume is not shown. B: Enable hardware accelerated texture mapping rendering. C: Enable shading, this emphasizes image edges.D: Enable MIP mode. When not using texture mapping the rendering is done ``in software''. In this case the ``Rate Controls'' (E:) appear in the control and can be used to control the frame-rate for rendering (and hence the quality). ``Rate 1'' controls the rendering rate when moving the volume and ``Rate 2'' when the display is static. (0.0 is the best quality). The Reslicing controls F: and G: specify the image resolution to be used for rendering (and the interpolation mode to achieve this). A higher resolution results in better rendering quality at the expense of slower rendering. The Cropping controls H: can be used to shrink the volume. Finally the Colormap controls I: are used to control color mapping.

6.4 Oblique Slices

While images are often looked at in orthogonal slices, sometimes there is great benefit in slicing the image in a direction that is not aligned with the acquisition axis (e.g. along the hippocampus). BioImage Suite has an oblique slice tool for doing this. The slice can be either manually positioned by specifying the normal direction and offset or by making it follow the camera position.

``Automatic'' positioning works by placing the slice perpendicular to the current viewer angle, this is enabled/disabled using the Follow Camera checkbox, which adjusts the plane in real time to follow the viewer camera. Alternatively the Look to Camera button may be used to perform this once. The Edge Camera button places the slice at the front of the camera clipping range (see the Renderer Controls for an explanation.)

The image display is controlled by (i) the main colormap – this is the same as for the main viewer (see Figure 2F), (ii) the resolution at which the slice samples the underlying image (this is set by the Res: drop menu – 256x256 is the default), and (iii) the opacity which can make the oblique slice more or less transparent – this is controlled by the Opacity slider.

**Figure 6.8:** **The Oblique Slice Control.** Top Row (A): The ``Show'' checkbox determines whether the oblique slice is displayed. If the ``Normal'' checkbox is enabled the plane normal is also displayed. If the ``Follow Cam'' checkbox enables automatic positioning perpendicular to the camera. The ``Res'' option menu (B) determines the image quality. The manual positioning controls (**C,D**) can be used to position the oblique slice ``by hand''. The opacity control (E) can be used to make this more or less transparent. The bottom row (G) has buttons for loading/saving the plane, the current 2D image slice (as an image) and the transformation (Save Xform). The later one can be used in BrainRegister to reslice the whole 3D image into this orientation. Additional facilities for manipulating the camera can be found in the row marked as (F) - these are discussed in the main text.

6.5 The Animation Tool

The animation tool is divided into two tabs. The first part, ``Main'' is a simple automated JPEG grabber. Just as you can save a single snapshot of everything in the black box viewing area using File - Export, you can use the Animation Tool's main function to save multiple JPEGs over a long amount of time with a given interval. To do this, select your delay time (or interval between snapshots) using the pulldown menu. Select your path using the ``browse'' button. You can edit the base name ``grabbedname.jpg'' to alter the resultant filenames (results will be e.g. grabbedframe000.jpg as shown in Figure 6.9). After pressing ``Start Grabbing,'' you can manipulate the image as you desire. Then press ``Stop Grabbing'' to terminate the process.

(note that in order to get the 3D animations you may be interested in seeing, you must set the viewer to ``3D only mode,'' which can be done in the viewer or using the convenient button at the bottom of the Animation Tool)

**Figure 6.9:** This image shows a snapshot of the animation tool which allows for automatic grabbing of frames from the viewer or manual grabbing of frames.

6.5.1 The Script tab

Scripting is a more advanced method allowing for interpolation of animation between frames. This will save you the difficulty and time of having to drag the mouse at certain speeds and directions to obtain an adequate animation. Figure 6.10 shows a step by step description of the ``Script'' tab which allows for the creation of detailed animations.

**Figure 6.10:** The Script tab. This tab allows for fine tuned controls for creating animations. The camera positions, duration at each camera position and the smoothness of the animation desired are some of the things that can be controlled using this tab. Section 6.5.1 has a more detail description for this image.

The script. This is comprised of several reference frames provided by the user. The tool will create a sequential animation from the first frame through each subsequent frame based on the parameters provided by the user. You can select frames using the up/down buttons or by clicking on them, and delete them individually or completely as necessary.
A readout of coordinates and information about the current frame selected, including the camera position and view.
A relative duration for the animation from the current to the subsequent frame for example, changing this value from 1.0 to 2.0 will make the animation take 2x the time, or operate at $\frac{1}{2}$ speed.
Interface between the viewer and the script. ``Add'' takes the current view in the viewer and adds it at a frame (in this case the next frame would be CM_35). ``Update'' refreshes the currently selected frame in your script to represent whatever is currently in the viewer. ``Send to viewer'' refreshes the viewer to show the currently selected frame. ``Auto update,'' when selected, sends the frame to the viewer as soon as it is clicked (eliminating the need to click ``send to viewer'').
Duration and subdivision factors for the entire animation (as opposed to the control in 3, which is for one step only). The subdivision controls the number of steps the animation takes to go from one frame to the next. Duration controls the length of the entire animation. Entering a low number of subdivisions and a length duration creates a ``slide show'' effect. Increasing the number of subdivisions creates a smoother, more detailed animation.
Animate shows what your current animation will look like based on the script and the parameters you have entered.
Calls viewer controls, or switches the viewer to 3D mode (most of the time the animation tool will be used to create 3D animations.). You may also want to use the ``volume'' drop-down menu in the viewer's right pane to flesh out the image in 3D.

6.5.1.0.1 Setup Menu

The setup menu allows you to load and save scripts for your convenience and exit the animation tool console when necessary.

6.5.1.0.2 Spin Menu

The spin menu is located within the Animation Tool and can be accessed as a drop-down next to the Setup Menu.

**Figure 6.11:** The Spin menu automatically computes and inserts camera positions that are 30 degrees apart. This prevents the need to accurately specify each and every position. Spin 10 inserts more camera positions which allows for smoother transitions between camera positions.

Spin 10 - automatically spins the model, horizontally, inserting 35 steps
Spin 20 - 18 steps
Spin 30 - 12 steps
Positive Rotation Adjusts between clockwise and counterclockwise spinning. The default, when positive rotation is checked, is counterclockwise.
Azimuth adjusts between rotation about the z-axis and the xy-plane. The default is azimuthal spinning on the z-axis.

6.5.2 Making a simple animation

Start by loading the Orthogonal Viewer from the main menu. Once the viewer has loaded, choose an image from File-Sample Images. In this case, we'll use the MNI T1 1mm stripped dataset.

Locate the Animation Tool from the Display dropdown menu as shown in Figure 6.12. The animation tool defaults to the ``main'' tab. Ensure that the save path is directed to an appropriate folder so that you can find the output files later. Experiment with this tab by setting the Delay time to 500 ms, and grabbing a few pictures with the ``Start Grabbing'' button while you rotate the image in the viewer. Click ``Stop Grabbing'' and examine the images in your file path. Move to the script tool using the tabs at the top of the Animation Tool window. Try making a simple script by clicking Add, then rotating the image slightly and clicking Add again. You may want to delete the original frame which will be called ``none'' as well. Once you have two or three points, click ``animate'' and view your animation in the window. Add a few more frames. Try doubling the default duration value for one of the frames to 2.0 and run the animation again.

**Figure 6.12:** The animation tool can be invoked by clicking on the ``Animation tool'' button in the ``Display'' dropdown menu.

Examine the Spin tab. You can either delete all of your first frames or add on to the animation. Use Spin 30 as it goes the quickest, and re-run your animation. Try changing the rotation direction and axis with the Azimuth and ``Positive Rotation'' options in the menu.

Once you have familiarized yourself with the options, you can choose to save your animation by clicking the ``Save while Animating'' tab next to the Animate button and run the animation again.

Next: 2 B. Anatomical Image Up: 1 A. Overview Previous: 5. Looking At Images Contents