5.7. Conclusion and Future Work

We have presented the current state of the tensor voting framework, which is a product of a number of years of research performed mostly at the University of Southern California. In this section, we present the contributions of the framework to perceptual organization and computer vision problems, as well as the axes of our ongoing and future research.

The tensor voting framework provides a general methodology that can be applied to a large range of problems as long as they can be posed as the inference of salient structures in a metric space of any dimension. The benefits from our representation and voting schemes are that no models need to be known a priori, nor do the data have to fit a parametric model. In addition, all types of perceptual structures can be represented and inferred at the same time. Processing can begin with unoriented inputs, is noniterative, and there is only one free parameter, the scale of the voting field. Tensor voting facilitates the propagation of information locally and enforces smoothness while explicitly detecting and preserving discontinuities with very little initialization requirements. The local nature of the operations makes the framework efficient and applicable to very large data sets. Robustness to noise is an important asset of the framework due to the large amounts of noise that are inevitable in computer vision problems.

Results, besides the organization of generic tokens, have been shown in real computer vision problems such as stereo and motion analysis. Performance equivalent or superior to state-of-the-art algorithms has been achieved without the use of algorithms that are specific to each problem, but rather with general, simple, and reusable modules. Results in other computer vision problems using tensor voting have also been published in [63, 30, 64, 69, 68, 67, 27, 28]. A major axis of our future work is the generation of tokens from images in a more direct and natural way. This will expand the range of problems we currently address without having to develop new algorithms for each problem.

Arguably the largest remaining issue is that of automatic scale selection or of a multiscale implementation of the tensor voting framework. It has been addressed in [69] under circumstances where additional assumptions, such as the nonexistence of junctions, could be made. Scale affects the level of details that are captured, the degree of smoothness of the output, the completion of missing information, and the robustness to noise. Since these factors are often conflicting, the use of a single scale over a complicated data set is a compromise that fails to produce the best results everywhere. One way to address this is to adapt the scale locally according to some criterion such as data density. Alternatively, a number of scales can be used and the problem can be transformed to one of finding the way to integrate the results over the different scales. A fine-to-coarse scheme was adopted in [69], which is consistent with the majority of the literature, but one could envision coarse-to-fine schemes as well. In any case, the issue remains open and is further complicated by that fact that even human perception is a function of scale.