Applying Geometry and Color Correction to Tiled Display Walls…

         Applying Geometry and Color Correction to Tiled Display Walls
              Justin Binns, Gennette Gill, Mark Hereld, David Jones, Ivan Judson, Ti Leggett,
                 Aditi Majumder, Matthew McCrory, Michael E. Papka, and Rick Stevens
                                       Argonne National Laboratory
                              {hereld,majumder,papka,stevens}@mcs.anl.gov

Introduction
Tiled displays systems built by combining the images from arrays of projectors can provide a huge number
of pixel elements to applications needing to visually represent a large amount of information, including a
range of scientific visualization applications and high-fidelity collaborative environments. With commodity
projectors and COTS rendering clusters, these large-area displays have the potential to provide very high
performance and high resolution at low per pixel cost. But it is difficult to create the illusion of a unified
seamless display for a variety of reasons. These include small misalignments of the projectors, optical
distortion of the individual projector images because of imperfections in the lenses, intensity variation from
projector to projector and across the field of each single projector, and color differences between projectors.
Argonne has been researching methods for addressing these limitations, in particular, geometry and color.
These methods are briefly discussed below. Both methods produce correction values that can be applied to
the projected images to create a single seamless image. For these techniques to become truly useful,
however, they must be integrated into existing infrastructure so that all applications and users can benefit
from them. Such integration is the subject of the rest of this paper.

Geometry and Color Correction Techniques
Hereld et al. have developed a technique for quickly and accurately assessing and correcting the alignment
and distortion of the tiled array using a commodity digital camera [1]. In our approach we take a few
pictures of test patterns covering of the entire display. Automatic analysis of these extracts a detailed
geometrical mapping to subpixel accuracy of projector pixels to the display surface coordinates. We use the
mapping to remove the distortions from the image by applying a piecewise bilinear warp.
Majumder et al. have solved the second piece of the calibration puzzle by introducing a camera-based
method for correcting the luminance field of the entire display [2-4]. Camera images of the display
projecting constant color values are analyzed to produce two quantitative measures of the system. First, for
each projector of the color output is remapped as a function of intended input value, to correct the projector
to projector color differences. Second, a smooth attenuation map is generated in the form of an alpha mask.
This mask balances the brightness of the display, and it provides smooth blending across overlapping
projected subimages.

Real-Time Geometry and Color Correction
Using the correction tables provided by the two systems described above as input, we have developed a set
of functions for applications that use OpenGL. One solution is for application developers working directly
with tiled displays. This library of functions for developers is currently implemented to work only with
OpenGL and NVIDIA extensions [5] but could easily be extended to work with other extensions and
DirectX. The second solution uses the functions just described to construct a Chromium SPU. Chromium
is a system for controlling and modifying streams of OpenGL API commands [6] and is commonly used for
the displaying of OpenGL applications in a tiled display environment.
To implement a solution that does not adversely affect the frame rate of the application, we use the latest in
OpenGL extensions. The process is as follows:
     1. The scene is rendered as it normally would be, including the use of vertex programs, texture
          shaders, and register combiners.
     2. The framebuffer is then copied by using glCopyTexSubImage2D to texture memory for the color
          correction step.
     3. Color correction is then applied by using texture shaders and register combiners, with the color
          correction represented as a texture map and alpha mask applied for overlapped projection systems
          (the alpha mask step does not need to be applied for nonoverlapped tiled displays).
      4. The resulting image is then texture mapped to surface where the geometry correction is applied by
         manipulation of texture coordinates.
     5. The final image appears on an individual tile of the tiled display.
This approach as been applied to a number of test cases. Table 1. shows the results of the various stages of
the process, as well as the overall impact on the system.
Table 1 Results for three different applications, showing the unmodified frame rate, render to texture (RT),
render to texture with color correction (RTCC), render to texture with geometry correction (RTGC), and
complete system with render to texture with color correction and geometry correction turned on
(RTCCGC). These tests all have the alpha mask step turned on.
Application           Unmodified           RT               RTCC              RTGC             RTCCGC
Solar System           89.3 fps          55.5 fps           48 fps           54.5 fps           45 fps
Atlantis               78.5 fps          63.2 fps          56.0 fps          61.5 fps           54 fps
Teapot                 153 fps            91 fps            72 fps            89 fps            70 fps

Conclusion
This work has demonstrated that it is possible to correct both geometry and color errors in a tiled display
environment without severe impact on the applications performance. This work has also demonstrated that
it is possible to construct a Chromium SPU that allows users of the Chromium toolkit to benefit from color
and geometry correction without having to modify existing applications. Future work will include testing
the system with graphics adaptors that have similar capabilities to those of the NVIDIA adaptors, as well as
extending it in the lower level library to support DirectX in addition to OpenGL.

Acknowledgment
This work was supported in part by Department of Energy, under Contract W-31-109-Eng-38.

References
[1]   M. Hereld, I. Judson, and R. Stevens, "DottyToto: A Measurement Engine for Aligning Multi-
           Projector Display Systems," Argonne National Laboratory, Argonne, ANL MCS Preprint
           ANL/MCS-P958-0502, May 29 2002.
[2]        A. Majumder, Z. He, H. Towles, and G. Welch, "Achieving Color Uniformity across Multi-
           Projector Displays," presented at IEEE Visualization, Salt Lake, Utah, 2000.
[3]        A. Majumder, "Properties of Color Variation across Multi-Projector Displays" presented at
           Eurodisplay, Nice, France, 2002.
[4]        A. Majumder and R. Stevens, "LAM: Luminance Attenuation Map for Photometric Uniformity in
           Projection Based Displays," presented at Virtual Reality and Software Technology, Hong Kong,
           China, 2002.
[5]        M. J. Kilgard, "NVIDIA OpenGL Extension Specifications," 2002.
[6]        G. Humphreys, M. Houston, Y.-R. Ng, R. Frank, S. Ahern, P. Kirchner, and J. T. Klosowski,
           "Chromium: A Stream Processing Framework for Interactive Graphics on Clusters," presented at
           SIGGRAPH, San Antonio, Texas, 2002.